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ABSTRACT - 

The report presents findiVigs f^om a study of the 
extent to whi'ch psychoeducational testing reports were used in one 
Navajo district and on the Obstacles affecting their ^use.i Data 
included an analysis. of findings and recommendations in 321 
individual psychoeducational' reports , a structured critique by school 
personnel of 79 testing reports to evaluate their educational 
relevancy, usability, and accuracy; an analysis of the relationship 
between report and Individualized Education Program recommendations; 
and observational and interview data.« Findings are presented on the 
three majoi; study objectives: identification of strengths and 
weaknesses of the psychoeducational system in use: design of an ' 
inservice training program for teachers, administrators, and school 
psychologists; and evaluation of results in both areas. The study 
resulted in six recommendations, including emphasis on converging 
data, obtaining Second opinions, and synthesizing reports to control 
cultural and language bias; design of a standard testing report 
format; and training and certif ica||ion as psychologists of Navajos 
who possess language and cultural skills. (CL) 
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Improving *the Utilization and Educational Relevance of 
Individual Psycho-Educational Assessment Reports in 
the Placement Of and lEP Development For 
Handicapped Native American Chlld^n 

^ IffrRODUCTION 

♦ 

Each year, schools throughout the United States spend millions of 
dollars on diagnosis and assessment of children with suspected learning 
handicaps (Goslin, 1963). A significant amount of this assessment is 
provided by school psychologists, admini stering individual psycho-educational 
tests to children who have been referred. Individual psychological assess-, 
ment traditionally includes three areas: (a) data collection that includes 
review of referral and existing assessment information, (b) assessment--the 
administration of individual tests designed to measure ability, lattguage 
skills, achievement, social behavior^ emotional factors, psycho-motor skills, 
etc., aild (c) communicating-reSults through preparation and submission of 
psychological testing r,eports and/or student staffings to assist in the 

n 

interpretation of testing results (Oakland & Matuszekj 1977, 1983). Irt 
practice, the abt)ve process may enl)3hasize one or two of the listed areas and 
virtually ignore others (Grubb, 1981). Depending on how completely each ^ 
component is addressed, the cost for each child's evaluation may range from 
$50 to several hundred. ^ 

There is evidence, ^lowever, which suggests that 'testing^ results, 
especially In the form of individual psycho-educational testing reports, are 

k 

not particularly useful to teachers« and other school personnel (Fifield, 
19S0). In many cases, individual psycho-educational reports are used only to 
meet the le^al requirements for placement of children in special education 
programs. If individual psycho-educational reports are not used to diagnose 



' problems and providing meaningfu> recommendations for remediation, the 

process of individual assessment becomes an irrelevant and extremely 
expensive exercise in both time and resources. Even though the utility of 
individually administered psycho-educational testing is a crucial issue, 
there is a paucity of literature which describes or documents efforts to 

^ systemal^ically improve the relevance and usability of individual^psycho- 
educational testing results. . ' * , 

This project investigated the extent to which testing report^ were used 
by school personnel in the Fort Defiance Agency on the Navajo Reservation, 
and identified the problems and obstacles which affect the value $nd the use 

of such testing reports. -Based on information collected in the first phase, 

( 

the psycho-educational evaluation and reporting process were modified in an 
0{ attenm to improve the utility and relevance of the assessment process in the 
placement of children in special education and developing individual educa- 
tion'al plans. As the second objective, the iwoject developed and implemented 
an inservice training program for school personnel and school psychologists 
to increase their understanding of individual tests and practice the skills 
necessary in using psycho-educational testing reports for placement and 
program development. The report describes the activities of the project and 

ft 

summarizes the evaluation data indicating the impact of the various • 
interventions. 

Funding for the Study , - . 

Support for the various components and ac;tivities which» constituted this 
study was obtained from three sources: ** 
1. A contract to the Exceptional Child Center from the Fort Defiance Agency 
of the Navajo Area Bureau of Indian Affairs Office to pro\?ide individual 

^ li 
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psycho-educational evaluations to Fort Defiance students referred during 
the 1980, 1981 and. 1982 school years. . 

2. A contract to the Exceptional Child Center from the Fort Defiance Agency 
• to provide inservice training addressing issues concerning psycho- 

4 

educational • testing, placement, referral, and programming decisions for 
V special education* chi Idren for 1981 and 1982. 

3. A grant from the U.S. Department of Education, Office'' of- Special Educa- 
tion, authorized under CFDA 84.023E Assessment Research. Funds from the 
Office of Special Education (OSE) were provided in response to a proposal 
submitted January 9, 1981 and approved to comnence September 3, 1981, 

♦ 

Grant #60081-00322. The OSE grant provided: (a) the staff to conduct 
the evaluation of the testing procedures and reports, including the 
design of the critique and evaluation instruments, ^^ervice training, 
and data collection, (b) computer time for analysis of data, and 
(c) preparation of reports. 
This final report is divided into the following sections: (1) Abstract 
and Summary of Recbrmendations, (2) Introduction, (3) Objectives and Method- 
ology, (4) Results, and (5) Discussion and Recommendations. 

Infection 5, "Discussion and Recomnendations", the results are 
interpreted, discussed, and recommendations to the Fort Defiance Agency and 
BIA Special Education Program are provided.^ This final section combines 
the quantitative data collected during the study with the perceptions of the 
project staff and the principals, special education staff, and special , 
service staff of the Fort Defiance Agency, These data were collected- prima- 
rily by Dr. Charles Deal^who throughout most of this study served as the 
Director of Special Eteation t)f the Fort Defiance Agency, and Mr. Norman 
Wilcox who served th§. last f^w months in this capacity. 




Accompanying this report is the "Inservice Ti^aining Resource Manual" 
which contains the inse»:vice training manuals used for the workshops provided 
Fort Defi ance .school personnel August 11-14, 1981 and August 11-12, 1982, • 
together with the workshop 'evaluation reports. -In addition, two inservice ^- 
workshops were tonclucted for the school "psychologists who administered the 
individual psychologica*t assessment to children referred during th§ years 
1981 and 1982. These workshops were provided August 27, 1981 Ind^^gust 28, • 
1982. |he agendas for these workshops, workshop materials, work activities, 
and exercises for- participants together with a^workshop evaluation report are 
also contained in the Resource Manual, 

If 
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ABSTRACT AND SUW1ARY OF RECOMMENDATKWS 

Atfstract 

It was the purpose of this project to: (a) determine, the strengths and 
weaknesses of the psycho-educational assessment process followed in the Fort 
Defiance Agency on the Navajo Reservation and to initiate appropriate changes 
■for improvement,- (b) develop and provide inservice training for teachers and 
administrative staff at the Fort Defiance Agency and to school psychologists 
who provide psychological services, ' and (c) evaluate the impact* of: .(1) the 
changes made in the individual psycho-educational testing,* and (2) the 
inservice train,ing provided to school personnel and psychologists. 

The research design utilized time series measures of assessment and 
intervention. Data was collected f rom 4hree in-tack schools in, the Fort 
Defiance Agency dOV-ing 1980 through 1982. The data included: (a) an 
analysis of th4 findings and recorrmendati ons contained in 321 individual 
psycho-educationH^^ports, (b) a structured critique by schooj personnel of 
79 testing reports to evaluate their educational relevancy, usability, and 
accuracy, (c) an analysis of the relationship between the recommendations in 
psycho-educational testing reports and recoimi^ndations on the students' lEPs, 
and (d)' observational and interview data collected systematically during t.he 
project. ■ , 

The findings, suggest that the utilization and educational relevance of 
psycho-educational testing reports can be significantly improved by: 

1. Increasing the familiarity of psychologists with the instructional 

¥ . • f. 

materials, service and placement programs, and teaching philos(J|!)hies of 
the school district., the building, and the individual teachers for whom 
testing reports are being prepared. 

■ . M . 14 
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2. Increasing the interaction and dialogue between the psychologist and 

/ 

appropriate sdT^ool personnel before testing and particularly in sj:afrfing 
iifter -testing. » 

3. Adapting the format of individual psycho-educational testin-g reports to 
ensure that ^y: include essential data qjoints necessary for placement 
and programming decisions, provide a synthesis^bf available student 

V * » 

assessment information, highl ight^ important data, eliminate jargon, and 
focus* on instructional recommendations. 

4. Improving and expanding the descriptive information contained from the\ 
student referral process. ^ • . 

5. Evaluating and providing feedback to the psychologists from schsol • 
personnel concerning the usefulness and relevance of their . assessment 
services in making placement and programming decisions. / 

-AM* 

From the .data collected and analyzed and the experience in conducting 
this study, the following recommendations are presented, and discussed: 

1. That the process by which students are referred for assessment services 
be reexamined and evaluated. ' 

2. That a "standard" testing report format be designed and utilized based on 
the- information needs of test result users and the provisions of Public 
Law 94-142. 

3. ' That culture and language bias be controlled procedurally. 

4. That Navajos be trained and certified as school psychologists. 

5. That future psychtD-educational testing contracts increase the specificity 
and the description of the services to be provided and that test result 
users participate in the selection of the contractor. 

6. That the instructional program in basic mathematics and reading for the^ 
Fort Defiance Agency be reexamined. 



Summary of Reconwendatlons * 

Jhe" recommendations and findings derived from'this study, including the 
data' collected and analyzed; the experiences of providing the psychological 
services, designing and conducting the inservice training, and evaluating the 
relevance of thfe testing reports; along with the inservice' training, are ' 
particularly applicable to the' Fort Defiance Agency. However, the findings 
also have direct application to many other BIA boarding schools where 
conditi^s and responsibilities are very similar. In addition, it is felt 
that the recomnendations and f*fndings will have meaning and application in 
regular public education and school districts in general. The following 
section presents a summary of the recooBnen^ations, further discussion of 
these recommend at it) ns is contained in Chapter* 5^ pages IIV through »12S. 

. / ' ■ r 

1. It is reconmended that the student referral process and forms by which 
. <? 

students are referred for psychological testing be examined and 
reevaluated. The referral process should serve both as a surmiati.on of 
, past concerns and as an entry point into the psycho-educational , 
assessment process. Referral information must contain certa-in provisions 
whereby the referral question (need or reason for the referral) can be 
clearly identified, thus enabling the diagnostician providing individual 
assessment to respond to the information needs of the referrer. If 
individual assessment is to provide mor^ than general orientation type 
data, it must serve the information needs of the referrer. When such 
need^ and concerns are not clearly presented, diagnosticians tend to use 
screening and broad purpose instruments which often duplicate what has 
previoihsly been done and do not provide the desired degree of specificity 
which makes information they contain relevant and usable for the, 
referrer. 

• .1 16 
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To control culture and language bias in psychological assessment, it is; \ 
recommended thaf convergent data procedures { conf inifation data and 

obtaining second opinions) and synthesizing , al 1 relevant data in a report 

* ■ . v 

should be folljowed. Existing indivi4ual assessment instruments do. not ; 

adequately control for culture bias and avail.able alternatives to " v 

standardized tests havs not proven to be effective. At this time, the 

-'most effective control for instrument and professional bias appears to be 

procedural safeguards emphasizing a variety of different instruments 

designed to obtain needed specific information with results val idated 

against other data to test out causative and treatment hypothesis. 

\t is recommended that a "standard" testing report format be designed and 

utilized based on the information needs of test result users and the 

provisions of Public Law 94-142. The ^'standard" testing report format 

should contain the essential' data upon which placement and program 

decisions can be based. The report format should require the examiner to 

either address each essential information point or provide an expfanatton 

as to why such information was not considered relevant. s^, ' f 

There is an urgent need for trained and certified Navajo psychologists 

fluent in Navajo and familiar with the culture to provide needed 

psychological services on the reservation. Until competent Navajo 

psychologists are available, the following recommendations are made: . 

a. Anglo school psychologists, in training, should be given greater 

opportunities to work with Navajo^ and Native American children under 

appropriate supervision. The current practice df contracting for 

psychological assessment services with only licensed psychologists, 

irregardless of their prior experience and familiarity with the 

Navajo way of life and the service delivery systems on the 
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reservation, prevents Anglo psychologists from obtaining the 
experience they need tp better serve Navajo children, 
b. When Navajo interpreters are utilized, theynnust be carefully trained* 
.and familiar with the concepts of standarili zed instructions and 
responses. 

5. Future psycho-educationaV testing contracts awarded to individuals or 
ag'encies should specify the following: (a) a standard psychological 
testing report format consistent with Recommendation 3, (b) staffing arid 
systematic interaction both before and after the testing is completed 
between tester* and school personnel, (c) orientation and training for the 
psychologists who are, awarded contracts to acquaint them with the service 
options, materials, And techniques available to serve children, (d) input 
from the test report us'ers in the selection of individuals or groups to 
provide^ assessment services, and (e) a systematic evaluation of each 
dimension of the testing process. \ 

6. It is recommended that the instructional program\^the Fort Defiance 
Agency be reexamined to determine if sufficient emphasis is being placed 
on Instruction, drill and practice to ensure that each cKfld achieves an 

\ 

\ 

acceptable degree of mastery in the basic skills of readingX^nd math. In 
analyzing the performance of 598, children from the Fort Defiance Agency 

who received individual assessment during the conduct of this study, 

J' 

virtually every child, rather eligible or nat for special "education 
services, was found to be significantly delayed in the primary skills of 
word attack and mathematics. Furthermore, many dedicated and concerned 
teachers and. admini strators expressed the concern that basic reading and 
mathematic skills were not receiving the emphasis they needed. 
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OBJECTIVES AND HETH0DOL66Y \ / 

It was the purpose of this groject to improve the utilization and 
educational relevance of individual psycho-educational assessment reports Ifi • 
making decisions concerning placement and in developi'^ng Individual Education 
Programs (lEPs) for Native American ^ildren. y/re specifically, the project 
was designed to: » ' * 

'1/. Determine the strengths and weaknesses in the psycho-educational 

assessment process followed in the ^rt Defiance Agency and to initiate 

appropriate thaages to improve it. 

2. Develop and provide ifiservice training to fche teaching and administrative \ 
staff of the Fort Defiance Agency and to school psychologists who provide 
psychological services to the Fort Defiance Agency to expand their 
knowledge and Skills in the utilization of individual psycho-educational 
assessment data. 

3. Evaluate the impact of: (a) the changes made in the individual psycho- 
educational testing procedures and processes, and (b) the inservice 
training provided to school personnel and psychologists. 

To accomplish these objectives, an assessment, intervention, 
post-aasessment, time series research design was utilized (Campbell & 
Stanley, 1963): 

oi xi 02 h 03 j 

0 = Observation (Use and perceived relevance of reports) 
X » Intervention (Inservice training) 

Dat^ collection and intervention procedures were carried out with three 
intact schools (Chuska, Tohatchi, and Greasewood Boarding Schools). Data 

were collected to evaluate the psycho-educational testing reports that were 

I . -r* . 

provided for these three schools during the years 1980 through 1982. 

19 / 



In 1980, following the administration of individual psycho-educational 

T 

testing, teachers from participating schools critiqued the testing reports, 
utilizing a specially designed questionnaire and structured interview. I-n 

.addition, the relationship between the findings and recommendations- contained 
in student testing reports and student individilal lEPs was determined {0]). 
Similar data were collected following the 1981 and 1982 individual testing 

'{02, O3). • 

During the fall of 1981, inservice training was provided to school 
personnel of the Fort Defiance Agency (X^). The content of the inservice 
training was selected to address changes in 'skills and information needed by 
school personnel to use testing -reports more effectively. This need was 
identified from the evaluation data collected from the critique of the 
testing process and reports provided by teachers in the 1980 testing. 

following the inservice training, changes were made in the individual 
psycho-educational evaluation procedures and test report format, and in the 
fall of 1981, testing was again provided to children referred by the partici- 
pating schools. To improve the utilization and relevance of the testing 
reports, they were evaluated by school personnel of the participating schools 
in the spring of 1982 (O2). 

The Fort Defiance Agency requested that 1982 testing be provided in the 
spring to the Chuska School and in the fall to the Tohatchi and Greasewood 
Schools. Prior, to the commencement of school in the fall of 1982, another 
inservice training workshop was provided to the Fort Defiance school person- 
nel (X2). The content of this workshop was again based on data collected 

/in the spring of 1982 which identified weaknesses and changes needed to 

> improve the use of psychological testing. 
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Following testirvg in the fall of 1982, data to determine the utilization 
and relevance of testing were collected and evaluated by school personnel 
(O3). These data, together with that collected in Febr[uaryJ982 and spring 
of 1981, were used to det-ermine the impact of the study. 

This study was organiz-ed into three broad tasks: collection of data, 
inservice training and intervention, and determining impact. Each of these 
tasks contain three or -more research steps. T^ble 1 provides a diagram of 
the objectives, tasks, research steps, timelines, and slource of support. 

Baseline data was collected from the individual psycho-educational tests 
administered' by the ECC to Fort Defiance children in the fall of 1980. the 
inservice training in 1981 was provided under a separ^flfflfcn tract with the 




Fort Defiance Agency. However, the content of the trainij!^vi?»N based upon 
the analysis. of the data collected during the spring of 1981. These^ s1;eps 
were pre-project activities (0], Xf). ' . 

The Assessment Research Grant began on September .3, 1981. Following the 
administration of the fall 1981 psycho-educational tests by the ECC in 
September and October, data evaluating these reports were collected and 
analyzed in the spring of 1982 (Step 1.1.2, 1.2.2) (Oj). ■ 

The 1982 psycho-educational testing of students was divided into two 
settings. Students referred from Chuska were tested in the spring of 1982 
before the closing of school. Students referred from Tohatchi and Greasewood 
were tested in the fall of 1982 shortly after school commenced. 

The second set -of inservice training workshops and seminars were 
provided for school personneTon August 11-12,- 1982 for the Fort Defiance 
Agency and the seminar for school psychologists August 28, 1982 (Step 2.2.1, 
2. 2. 2, .2. 2. 3) (X2). Evaluation data for the 1982 testing were collected in 
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Table 1 » 

Assessment Research 
Objectives, Research Design, Timelines, and Source of Support 



Oh.ioct ives 



Project Tasks 



Pre-Project Activities 



r 



Determine Strenqths 
aiuLWeaknesses in 
StijJl^nt Assessment 
of the Fort Oef i anre 
Aopnc.y 



1.1 Collect Data (0^) 



?. 1 Provide Inscrvice 
Traininq and 
Intervent ion (X ) 



Assessment Project Begins 



1.? Collect Data 



Develop and Provide 
loservice Traininq to 
Staff and Psvcholoqists -<f 
and Modify Testinq 
Procedures 



?.? Provide Inservice 

Traininq and 
intervention (X ) 



Evaluate the Inpact of 
Changes in Test Pr^^- 
cednres and Format and 
Inseryice Traininq 




3.1 Collect Data (0^) 



3.2 Analyze Data 
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Research Steps Timelines Sourcfe of Sunnort 


1.1.1 Administration of Individual Psycho- 

unuLaiioiioi lUbLing LO irfiiioren Keref reo 


1980 
beo - uct 


Fort Defiance Contract 


l.?.l Evaluation of the 1980 Psycho-Educational 


1981 

Ian t^K 

ijan - reo 


ttt Support 


?.1.1 Inservice Workshop 1, Fort Defiance 
oLnuu 1 rcrbuiiiir 1 , oLUQcnt Assessment 


August 


Fort Defiance tontract 


?. 1.1.1 Evaluation of the Workshop 


September 




2.1.2 Inservice Seminar 1, School Psychologists 


August 


ECC Support 


. 1 . o nuuiriLdiiun or inoiviQuai lesving 
Process and Report Format 
.. -^----A. : . ^ , .-^^^^ ^ _w_ 




ECC Support,, 


1.1.2 Administration of Individual Psycho- 

cuucaLiunai icbtiriu io^i,niiaren Kererrea 


bep - Uct - 


* 

fort Def i ance Contr 


act 


1.2.2 Eval+jation of the 1981 Psycho-Educational 
1 t-f b4. 1 ng Kcpor i s 


1982 
Jan - reb 


Assessment Researct 


■ 


2.2.1 Inservice Workshop 2, Fort Defiance 
^LHou 1 rcrbonnci , ipj tuocni. Assessment 


August 


Fort Defiance Contract 


2.2.1.1 Evaluation of the Workshop 






2.2.2 Inservice Seminar 2, School Psychologists 




Fort Defiance Contract 


P ? ^ MnH i 'F 1 r inn f\f TnHiuirliiAl Tocf inn 
c • f. m J nuu 1 1 iLatiuii ui i iiu i v i ijud ■ icsiing 

Procedures and Report Format^ 




Assessment Research 


1.1.3 Administration of Individual Psycho- 
Educational Testing to Children Referred 


Sep - Oct 


Fort Defiance Contract 


*3.1.1 Evaluation of the 1982 Psycho-Educational 
Testing Reports 


November 


Assessment Research 


3.2.1 Data Analysis to Determine Imnact 


Deceirtber 


Assessment Research 


^2^2 Prji|iajiaLiUm-u>£-Xi^ ^ 


4983 


Assessment Researcir 




March 
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\ ' ■ 

November of 1982 fStep 3.1.1) (03).- The, procedural phases, tasks, and ^ 

■ ■ ■ • 

\ steps for the project ai^ presented in greater detail in Appendix B. . 

Population and^ata Collection 

The population used in this study consisted of three groups of . people 
associated with the Fort Defiance Agency: students, teachers, and school^ 
psychologists. The Fort Defiance Agenty on tbe Navajo Reservation was 
selected forythis study for the fol lowing reasons: ■ • , 

-1. The Navajo Reservatipn is the largest -Indian reservation in the- United 
States. Although it may appear to be somewhat atypical of other reserva- 
tions and Indian populations because of its size and remoteness, many of 
the same problems that exist on the Navajo Reservation exist on other 
reservations and with other tribes. Furthermore, the educational 
services provided, difficulty in training staff, and environmental 
factors on the Navajo Reservation are similar to other BIA schools across 

r 

the continental United States and Alaska. Havighurst (1981) reports that 
approximately 18X of the Indian children are being educated in BIA 
schools. In addition, 38% are attending public schools contiguous to 
reservations in v^ich from 50 to 90% of the enrollment is Native 
American. Staffing and organizational arrangements for these boarding 
schools are very similar to those for BIA schools. 
2. The Fort Defiance Agency School Office has, over the past three years, 
developed a working relationship with t\\e Exceptional Child Center. 
Previous psychological testing contracts from the Fort Defiance Agency to 
the'^Exceptional Qiild Center provided an opportunity to acquaint the ECC 
St aff/wmfrte organization, philosophy, and resources of the Fort 
Defiance Agency.' This insured thet^needed familiarity with' the agency an(l 
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cooperation and acceptance of psychological services from the Exceptional 
"child Center. » 
Within the Fort Defiance Agency, three schools (Chuska, TohAtchi, and 
Greasewood Boarding Schools) were identified to participate in the project. 
These schools were selected as being representative of the geographic and 
social economic conditions that- exist on the Navajo Reset-vation. They are 
typical of boarding schools in the Fort Defiance Agency and indeed of other 
boarding schools serving Native Americans. 
Parti clpati rig Student Population 

The participating student population included students referred for 
psychological^assessm6nt from the Fort Defiance Agency and tested during 
1980, 1981, and 1982. Tabie 2 7'ists the number of children referred and 
tested from the target schools and those from the other schools in the 
Fort Defiance Agency. 

The participating student population included those students 
attending the three target schrools in which data were collected ^and 
intervention techniques and procedures initiated. 
Data*Co11ccted From the Student Population 

I 

The content of the psycho-educational testing reports for each 

I 

student referred and tested in the target schools in the Fort Defiance 
■ Agency was analyzed and the results surmiarized and tabulated. This 
information is presented in Table 3, page 29. From the target group 
studeftt population, data was also collected to determine the relation- 
ship between recommendations and findings contained in the testing 
•reports and those contained on the students' lEPs. A sample of 
psychological testing reports were abstracted from this group. These 
reports were then critiqued by school personnel familiar with students 
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FIGURE 2 * 

.\ges and number of children tested 

1980-1982 

N 

/ 

100 ■ — : r ^ : 




■■1 TARGET SCHOOLS 333 CHILDREN TESTED 
CZZ3 OTHER SCHOOLS ' ^5 CHILDREN TESTED 

|Pii8 TESTED 

> . " 26 

■■■ 16 " 
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for ■which the reports were prepared. It was this data that provided 
information on the relevance of the psychological testing report and its 
usefulness and the improvement that occurred through the inservice 
training and the changing of the testing format as evidence of the 
impact of intervention throughout the study (see Appendix C, Report 
Critique Forms). 
School Personnel 

The administrative, supervisory, and instructional staff at Chuska, 
Tohatchi, and Greasewood Boarding Schools served as the school personneV 
population for this study. A sample of school^rsonnel from the target 
schools was^ selected to critique the psycho-educational assessment 
process and provide evaluation data on the content and format of the 
i"ndividual psyctio logical testing reports. 'This sample of reviewers 
included regular teachers, special education teachers, administrators, 
supervisors, and counselors. The reviewers consisted of approximately 
of the certified personnel of the target schools and approximately 
5% of all certified personnel at the Fort Defiance Agency. To maintain 
confidentiality and to' insure that the school personnel who reviewed the 
reports were familiar with the child whose repor| they were a^ked to 
critique, the following procedures were utilizfedf 

1. A sample of testing reports from the t|jjpet schools was identified 
by random selection. 

2. From the testing reports selected', the school personnel >*io ^ ^ \ 
participated in the referral process, lEP development, and/or who 
had other responsibilities for the child were identified as the test 
report reviewers. Of those identified, approximately 75X in each 
school were regular teachers. This number of regular teachers 
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participating in the study changed slightly from year to year due to 
teacher turnover and as a result of the referrals they made for 
testing. The counselors, supervisors, and special educators were, 
virtually the same through the study. 
Data Collected From Reviewers 

Most of the evaluation data for this project were provided by the 
sample of school personnel who reviewed the psychological reports. The 
' reviewers utilized the Psycho-Educational Testing Critique Form to 
critique the individual testing reports, participated in the inservice 
training, and were Interviewe/^indlvldually at the beginning, at 
midpoint, and at the conclu/on of the project. 
School Psychologists 

The school psychologists who participated in the study consisted of 
19 advanced graduate students and faculty members from Utah State 
University. Each psychologist selected to participate in this stua/ 
had completed appropriate graduate course work ahd practicun experience " 
prior to the study. Furthermore, each psychologist was personally 
recommended by faculty members and practicum and internship supervisors 
•as being skilled in administering, interpreting, and reporting 
individual psycho-educational tests. In addition, each graduate student ' 
was supervised by faculty members and his or her team leader. 

Instruments ~~' . 

The Instruments utilized in this study consisted of commonly used 
individually administered standardized tests, instructional objectives ^ ' 
contained in the STEP manual (see page 14). a summary data sheet, a 
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psycho-educational report critique form, and a structured interview, each 

designed specifically for the study. 

Individually Administered Standardized Tests 

>^The standardized tests selected to provide psycho-educational 
assessment of children referred included several widely utilized and 
accepted testing instruments. These tests were selected to provide 
measures in the following areas: ability, language skills, achievement, 
social behavior, emotional factors, psycho-motor skills, and adaptive 
behavior. 

Appendix, D provides a list of the tests and the frequency of their 
use for the children referred from the target schools and other schools 
in the Fort Defiance Agency. A one-page- descriptor of each of these 
tests is contained in the inservice training manual for school 
psychologists, pages 117 through 121. In addition, a single-sentence or 
short- paragraph descriptor of each test was included in the individual 
testing report, page 317. 

Utilizing individually administered psycho-educatignal tests 
consists of more than simple test selection. It is a process of: 
(a) understanding the reason for the referral, (b) obtaining existing 
information, '(c) matching the need for new information with appropriate 
assessment instruments and other techniques, (d) administering assess- 
ment tnstruments, (e) comparing findings and results with concurrent 
data, (f) verifying conclusions, (g) synthesizing information from 

■ 

different sources,* and (h) communicating conclusions and recoimenda- 
tions. 

This study utilized an adaptation of the "dispositional assessment" 
model as outlined by Cole and Wagnussen (1966). The following 
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factors were considered in determining assessment procedures and 
selecting instruments: 

1. Where necessary and to the greatest extent possible, psychologists 
:,utilized standardized tests purported to be unbiased, culture fair 
and non- language dependent. 

2. Where possible, criterion-reference tests were utilized to identify 
specific skills, strengths and weaknesses. ' ^ 

3. Where possible, adaptative behavior measures were utilized, teacher 

rating scales were used fbr all children referred, and responses to 

special behavior rating scales were obtained from counselors and 

dormitory supervisors. 

. / 4. Tests were selected and assessment procedures utilized to focus on 
¥ — ^.^^^^^^^^ f 

the reP^^isilauestion. 
5. No test was adirg^vjstered unless its results would contribute an 
appropriate decisiayi or action. 
^ 6. The assessment process always involved more than one person. All 
protocols and test ingXinformat ion were presented to an outside 
certified school psychologist for a second opinion. 




7. All assessment findings 



were synthesized into a report prepared with 



non-jargoni Stic languag< , presented in a manner that the chi 

parents' could read and Ajnder stand the report. 

Additional informaj'fon on the procedures of testing, training of 

school psychology strand evaluation information on the individual 

testing conpact^s available in "Psychb-Educational Assessment of 

Nat'ive Americai/ Students: A Manual of Inservice Training Activities and 

Resource Materflal for School Personnel and School Psychologists", il 983. 

i \ 

This documenttis a companion to this report. 
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STEP Objectives for Student lEPs 

f 

In 1979, the Fort Defiance agency*^a^opted the Sequential Tasks for 
Educational Planning (the STEP Program) \s the basis for developing 
student lEPs (Greenberger , 1977). The program provides a single page 
listing of functional diagnosis and a task analyzed sequence of 
measurable job jectives and was developed '^d published by the Kajon 
Valley Union School District. For this study, the STEP Program was 
utilized in the following manner: once a student had been determined 
eligible for placement in special education by the psychologists,' the 
lEP Committee met to consider the findings and recommendations in the 
testing report and review other ''V-ecords. These records included the 
student cumulative folder, health records, attendance record, and social 
and "academic records. When necessary, donn supervisors, aides and other 
school staff were invited to the lEP meeting. In all instances, efforts 
were made to include the parenC-or guardian of the child. The lEP 
Committee reviewed alternative placements for the child, and in cases 
where they felt special education pi acement- was warranted, such a 
recommendation was made. At this poirij;, committee members selected the 
specific task number and activities from the sequential ' objectives that 
they felt met the child's instructional, social, artd behavioral needs. 
These objectives were listed on the student's lEP as short-term and, in 
some instances, long-term goals, and the corresponding number of the 
task recorded. 

Psycho-Educatlonal Testing Report Format 

Psychologists use a variety of methods to communicate findings and 

I 

recommendations of individual tests to school personnel. The most 
cofnmon techniques are direct consultation', either individually or in 
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groups (staff ings), or through psychological testing reports. Although 

« 

much has been written about the need to improve methods of communicating 
testing results, there are no accepted guidelines (Sattler, 1982). 

When a school district contracts with an "outside" individual or 
agency for psychological testing services, the opportunity for the 
psychologist to communicate test findings through consultation with 
school personnel is limited. Thus, testing results must be communicated 
primarily through Vhe use of psychological testing reports. 

One problem in conducting research using psychological testing 
repprts is that test report formats vary tremendously. Some test 
(reports consist of only the original test protocol upon which the 
psychologist has written brief notes, while others contain Several 
sections, each addressing specific information needs and synthesizing 
information from various sources. 

The contract awarded "to the ECC to provide psychological testing to 
the For f -Defiance Agency specified a detailed synthesized psychological 
report that addressed the areas of information required by Public Law 
94-142. The format of the testing report was submitted to and approved 
by the Font Defiance Agency and the U.S. Department of Education (see 
Appendix E). - 

PsychO'Educatlonal Testing Report Critique Form 

To collect data from school personnel to determine the usefulness 
and relevance of the testing reports, a Psycho-Educational Testing 
Report Critique Form was 'designed. The Psycho-Educational Testing 

Report Critique* Form was initially developed and field tested in a 

• ■ V 

previous study^ published in 1982 (Fifield, 19.82)^. This form was'revised 

* 

and adapted for utilization in this study. 



The Report Critique Form consisted of four parts: the first two 
parts collected general inforfnati on about the content of the psycho- 
logical reports, and the third and fourth parts requested respondents to 
rate verbatim quotations from reports- selected as a sample of testing 
reports prepared for, and provided to the target schools (see Appendix C 
for a sample of the Psycho-Educational Testing Report Critique Form). 
The questions in Parts I. and II of the report form were the same for all 
students in iihe sdhiple and focused on the clarity and technical accuracy 
of the reports. The questions in Parts III and IV were designed to 
determine if the information was communicated clearly, if it was 
correct, and if the conclusions were useful in planning the student's 
educational program. To collect this information, a sample of reports 
from each phase of the study was selected and analyzed. The conclusions 
and reconmendationis made by the psychologist were taken verbatim from 
the report and placed orv the Report Critique Form adjacent to the rating 
scale. 

Part I - Adequacy of Information . Part I of the Report 
Critique Form afked reviewers to evaluate^e six dimensions of the 
testing report: (a) the extent to which the report clearly stated 
the student's testing results, (b) the extent the report was felt 
to be useful in determining the student's placement, (c) the extent 
to which the reviewers found technical words and phrases, difficult 
to understand, (d) the overall quality of the report compared with 
reports the reviewers had seen in the previous J^sar, (e) the extent 
that the recommendations of the r,eport addressed the referral 
• " question, and (f) the extent the reviewers felt the psychologists 
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gave appropriate consideration to social and cultural factors 
essential to the evaluation of Native -American children. 

Part II - Usefulness . Part II of the Report Critique Form 
requested the reviewers to evaluate the seven sections of the 
psycho-educational report: (a) Referral Information, 
(b) Background Information, (c) Behavioral Information, (d) Psycho 
Educational Findings, (e) Diagnostic Statement, (f) Summary, and 
(g) Recomnendations (see Appendix C). School personnel reviewing 
these reports were asked to rate the extent that each section 
addressed the information requested in the report and to rate how 
useful the information was in that section in preparing students' 
individual educational program. 



Part III - Usefulness and Rel<^Vance o f the Conclusion . 
The usefulness and relevance of psychological testing reports are 
determined primarily by the content of the conclusions and 
recoimendations sections. Part III of 'the Report Critique F6rm 
requested reviewers to rate verbatim conclusions which the 
psychologist made about a particular studenj/^. This procedure made 
it possible for reviewers to rate each conclusion individually 
without having .to ref€r to the total report. 



Part IV - Usefulness and Relevance of the Recommendations> -^ 
The questions in P^rt. IV of the Report Critique 'Form were designed 
to determine: (a) the extent that the recommendations from the 
psychologists were at the proper level of specificity to be useful 
to school personnel, (b) the extent school personnel felt 
statements in the report provided new information, cpnfirmation of 



teachers' Judgements, or no new information, (c) how realistic the 
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recommendatijOris "were, given the conditions and resources available' 
on the reservation, and (d) how useful the reconwendations were to 
school personnel in developing the students' individual educational 
plan. - • 
Structured Interviw 

A structured interview was conducted with each reviewer focusing on 
various sections of the Report Critique Form. The interview prpvided a 
validity check of the Report Critique Form and an opportunity to collect 
informal Gormients about the process of individual testing and the ^ 
content of the reports. These data were used to develop the inservice 
training workshops and seminars, to clarify, follow-up on and gather new 
Information about questions raised in written questionnaires. « 
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RESULTS 

This 'section focuses on the three major objectives of the study as 
listed in the initial grant application to the U.S. Department of Education. 

1. Determine^ existing strengths and weaknesses in the psycho-educationaT 
assessment reporting system and process for the Fort Defiance Agency. 

2. Based on data collected in Objective 1, design and present an inservice 

training program for teachers and administrative school personnel of the 

Fort Defiance Agency and school psychologists. 
I 

3. Evaluate the impact of: (a) changes made in the individual psycho- 
educational evaluation process, (b) the test report format, and 

(c) inservice training provided to school personnel and psychologists. 
To further address Objective 1 , a series of questions were formulated 
and instruments and techniques were designed to collect needed data. To 

* 

address Objective 2 , inservice workshops were designed and provji^ed for 
school personnel of the Fort Defiance Agency and ECC psychologists^ Two 



separate workshops were conducted for each grou^, the first in \he summer of 
1981 and the second in tJi^ supner of 1982. The content of these workshops, 
handout materials, etc., are contained in the Resource Manual. To further 
address Objective 3 (Determining the Impact of the Inserv(ce''^raTiiing and 
Changes in the Psycho-Educational Testing Process and Testing Report), the 
evaluation data and information collected as baseline data was again 
collected early in 1982 and at the conclusion of the project in flovemb^r 
1982'. Comparisons were made to determine change. ''that occurred, the direction 
of such change, and its significance. 
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Objective 1 - Determine Existing Strengths and Weaknesses in the Psycho- 
tducatlonal Assessment Reporting System and Process for the Fort Defiance 
Agency 

Information identifying strengths and weaknesses of the 1980 testing 
process was obtained by utilizing three techniques: 

1. Analyzing the findings and recommendations- contained in the testing 
repots submitted for the target schools. 

2. Presenting 'a sample of psychological testing reports to scfiool personnel 
in the target schools and having them systematically critique the report 

« 

to identify the information that was relevant and useful and to what 
extent. , ' ' ^ ^ 

3. Interviewing school personnel to obtain their opinions and feelings 
concerning the value of psychological testing and how it could be . 

improved,. % ' 

The first task in determining the strengths and weaknesses of the 
existing psycho-educational assessment process necessitated establishing a 
common testing procedure and standard test report format to be used 
throughout the course of the study (see Appendix E, Psycho-Educational 
Testing Report Format). To obtain a standard testing report format, the 
requirements of Public Law 94-142 and the BIA guidelines for psychological 
services were analyzed. The format* agreed upon was designed to comply with 
the information required in these two documents. The psycho-educational 
tf^ testing procedures followed during the course of th;4is--study are contained in 

the Resource Manual , pages 283 through 286. f . 

During the spring of V981, the contents of the 1980 psychological 
reports submitted to Chuska, Tohatchi, and Greasewood were analyzed. To 
analyze these data, a summary sheet was prepared which permitted analysis of 



the 1 nf orma't 1 on contained in the findings and recommendations sections in 

It 

addition to a frequency count of these findings and recormiendations. 

Areas showing weaknesses were given particular attention in the 
inservice training, and/or adjustments were made to the testing report 
process or format for the next phase of the project^ The data collected for 
the 1-981 and 1982 testing were analyzed in the same manner as the 1980 data. 

Summary data of the findings and recommendation's for the target schools 
are presented in Table 3, pages 29 through 39. The categories in which 
findings and recommendations were grouped include: (1) placement, 
(2) short-term goals, (3) long-ter^ goals, (4) instructional strategies and 
materials, (5) support services, and (6) evaluation criteria for success. 
1 . PI acement ♦ 

The data presented in Table 3 indicate that approximately half of 
the students tested were found eligible for special education services 
under the classification of "learning disability." Over the three years 
of the project, the percentage of students classified as learning 
disabled remained fairly consistent. 

Approximately one-quarter of the students tested were found to be 
•not eligible for placement in special education, 20% in 1980, 27% in 
1981, and 29< in 1982. The number of students found not eligible for 
special education increased, but this increase was found not 
significant. 

The number of children the psychologists found eligible for special 
education services in the trainable mentally retarded cate^gory reduced 
over the three years^of the project; 19% were diagnosed as mentally 

•x> ^ 

retarded 'in 198o/l3% in 1981, and 9% in 1982. This change however was 
not Stat IsticaUy significant. The number of students found 'eligible 
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Analysis of the Findings and Recommendations Contained on the 
ECC Psycho-Educational Reports for Students Tested at the 
Target Schools for the Assessment RftstStKh Project 
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Cateqorles . 
Findinqs and Recommendations 
Taken From Testing Reports 
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No ^i^ement Recommendation/The Decision Was Not Definite 


0 


0 


16 


3 


0 


6 


13 


5 


15 


8 


0 


7 ^ 


. V 1 TOTAL 






77 

•5i-. •;i.'>;- 




79 


no 















for special education under the classification of emotional ly disturbed, 
. vision or hearing impaired, or other handicapping categories appeared to 
ini^inately low. The Director of Special Education for the Fort 
Def iance-;Agency pdinted out that children with health, vision, or 
physical impairments are qener^Mx not referred to the psychologists but 
are referred to the Indian Health Services and determination of their 
eligibility is made by IHS clinics. 

The most frequent placement recommendation for special education was 
for a resource room a. specified percentage of the day. Such a recormien- 
dation was given to 64< of the students tested in 1980, 43X in 1981, and f 
37% in 1982. The data on Table 3, page 29, indicate that during the 
course of the study, a greater frequency of alternative placements were^ 
recommended, particularly increased were recommendations for tutorial and 
peer instruction. 
ShortTTerm Goals 

One of the most important sections of the testing report addressed 

recommendations for short-term goals. This section is particularly 

. *- ' 

important because it is the short-term goals that are most often 
transferred to the students' lEPs. For the purpose of the testing 



report, short-term goals were defined as those recommending immediate 

\ ■ ^ • , ^ 

attention that* should be accomplished within one year. Psychologists 

were asked to recommend shoi^^term goals in the following areas: 

reading, writing and spelling, math, and others (see Table 3, page 31). 

The recommendations on testing reports for these goals were derived from 

the results of the tests administered. As a resuK? the area, the 

number, and the specificity varied for each child tested. Efforts were 

made to list the f^coimiendations for shori-term goals in a way that would 
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Table 3 (continued) 



Categories 
Findings and Recownendat ions 
Taken From Testinq Reports 


1980 


1981 


1982 


m o 

t 

t 


£ II 
^ Z 

% 


m ro 
S H 

% 


Total # of 
>e Rpts Reviewed 
N = 172 


1" 

X 


CO 

S. It 
F Z 

X 


^ 00 

g U 

0 Z 

% 


Total # of 
X Rpts Reviewed 
N «= 79 


m o 
in 

X 


S U 

^ z 
X 


2 " 

w z 

% 


o ^ 

« O 

fi ce z 
X 


2. Short-Term Goals 

a. Recommendations to Improve Read1n9 Skills 
General Improvement of Reading 


11 
11 


1 o 


ID 


13 


c- — ' — 




D J 


34 


ID 


o 
O 






^ ... - 

Word Attack 








55 




He 




35 




97 
C / 


1 7 


£0 


Phonics Skills/Pronunciation and Blending ' 












Jl 


2, 


20 




"^l 
Jl 


91 
£ 1 


c J 


Increased Letter Identification/ Sight -Word Vocabulary 
and Recognition, and Reversals 


72 


59 


61 


67 


43 


60 


25 


52 


80 


70 


46 


63 


Functional/Survival Reading Skills (Newspapers, Magazines, etc.) 


26 


13 


6 


20 


0 


2 


0 


1 


15 


15 


17 


16 


Improved Comprehension and Recall 


M 


28 


25 


20 


4 


23 


0 


15 


30 


27 


17 


23 


SUBTOTAL 


cm 


_i 


I OH 


209 


1 J7 


1 on 


1 1 


166 


CW 


1 77 
iff 


1 €i1 


1 on 


b. Recommendations to Improve Writing and Spelling Skills 
General Improvement of Spelling and/or Wrltinq 


13 


10 


3 


10 


47 


20 


13 


28 


10 


.15 


21 


16 


Specific Spelling Improvements (irregular Words^ Plurals, etc.) 


25 


21 


19 

1 7 


23 


n 


' A 




3 




n 


1 'X 




Writing Skills (Alphabet, Grammar, Hechanlcs, Words Md m 
. Sentences from Dictation, etc.) ^ft 




5 


16 


11 


17 


38 


13 


29 


10 


31 


21 


21 


•subtotal 


49 


36 


3^ 


44 


65 


63 


25 


59 


25 


46 


54 


43 


c. Recommendations to Improve Math Skills 
General Improvement of Math Skills 


13 


5 


13 


11 


17 


25 


50 


25 


15 


8 


21 


14 


Pre-Math Skills (Codnt1ng» Number Identification and Concepts) 


9 


3 


6 


7 


4 


6 


0 


5 


0 


8 


17 


9 


Addition Facts (Single and Multiple Digits) 


52 


85 


35 


56 


35 


31 


0 


29 


55 


31 


42 


41 


C 

Subtraction Facts (Singular and Multiple Digits, 
^ i^^luding Borrowing) 


74 


74 


35 


67 


61 


42 


_o 


43 


60 


-it 


50 


46 



Table 3 (continued) 



Categories 
Findings and Recommendations 
Taken From Testing Reports 



HuUiplication Facts (Single and Multiple Digits) 
Division Facts (Includes Long Division with Remainders) 



1980 



CM 
O 



26 



25 



u 

2 



49 



31 



32 



32 



4»< lU (N 



33 



28 



1981 



C4 



30 



35 



_ 00 

S N 

H as 
% 



27 



21 



25 



38 



28 



27 



1982 



5° 



50 



30 



\ * 

4 Z 

% 



19 



23 



0 2S 



42 



33 



Fractions (Includes Dectwals) 



16 



10 



Functional and Survival Math (Honey, Time, Heasu'rl^ment . 
Calculator Use, etc.) 



60 



46 



35 



52 



22 



40 



19 



50 



to 



d. 



44 



Story Problems 



14 



10 



T 



SUBTOTAL 



277 



300 



97 



271 



204 



152 



113 



163 



260 



142 



Other Short-Term Goals 

Improvement of English Language Skills 

English Reception 

English Expression , 



15 



23 



50 



22 



45 



27 



36 



10 



10 



26 



31 



27 



50 



23 



29 



33 



26 



35 



29 



28 



45 



23 



Auditory Discrimination Training and Improvement 
Visual-Hotor Training {lnc}^es Directionality) 



16 



23 



16 



16 



2S 



15 



25 



18 



16 



22 



22 



25 



13 



23 



Pre-acaaenlc Skills (Toileting, Dressing, Eating, 
Colors, Body Parts (Hygiene), etc.) 



13 



Vocational/Survival Skills 



20 



19 



267 



25 



erJc 



General Behavioral Skills (Includes Appropriate Class Behavior, 
Attention, Independence, Completion of Assignments, 
Responsibility and Attendance ^_ 



16 



10 



23 



16 



35 



16 



38 



24 



35 



23 



General Social Skills (Includes Self-Concept, Confidence, 
Interaction, and Participation) 



18 



13 



29 



19 



43 



33 



SO 



38 



50 



15 



46 



21 



SUBTOTAL 



125 



162 



90 



127 



187 



125 



175 



148 



270 



165 



113 



m 



Increase their uSefiilness to the lEP Committee. Thus, the recommenda- 
tions for short-term goals were, in most cases, referred directly t.o 
instructional objectives in the STEP Program.^ 

a. Reading 

The mo^t frequent recommendations made for short-term goals were 
for reading. Six categories were identified with word and letter 
recognition skil\s being most frec|uently recommended (see Table 3, 
page 31). Approximately two-thirds of the Children referred and 
tested had deficits in word and letter recognition skills signifjicant 
enough to warrant specific recoinnendation^ that the instructional 
program emphasize efforts to strengthen these skills. This emphasis 
was consistent throughout the course of the study. Recormiendations 
for additional instructional empKasis in comprehension and suryival 
reading was identified as a need much less frequently. These data 
suggest that the chsildren tested had not mastered basic and elemen- 
tary reading skills, thus, they were not ready to pursue intermediate 
Reading skills. Figures in the subtotal category of reading 
instruction suggest that the average testing report contained 
approximately two recommendations for short-term goals addressing 
this need. ^ 

b. Writing and Spelling Skills 

Recommendations for improving writing and spelling skills were - 
made on approximately 20X of the reports. , Writing skills were most 
frequestly recommended. |/ Specific spelling skills, i.e., irregular - 
words, plurils, etc., were reconmended in fewer instances, partic- 
ularly during the latter two years of the study. 

. 46 ( 

33 



c. Math 

The second most frequent recomniendations were for math. Of the 
nine categories of math instruction reconmended, instruction in 
, elementary mat hematic facts, including single and double digit 
addition facts, and subtraction facts, including borrowing and 
carrying, constituted two-thirds of the recommendations. The ^ 
distribution of recoimiendations aross math categories Was similar 
throughout the study. Intermediate mith skills were recommended in 
less than a third of the reports and the frequency of such recomnen- 
dations did not change significantly during the study. Figures in 
the subtotal category of math skills indicate the average testing 
report contained two or more recommendations for math instruction. 

d. Other Short-Term Recommendations 

Other frequent recommendations included instruction for 
improving the child's use of the English language. Although English 
language skills are a major problem on the reservation, it is noted 
that a significant amount of time and effort is directed- towards 
English language skill development in the curriculum. Some psychol- 
ogists diti not recommend additional instruction in this area for they 
were mindful of the amount of effort currently devoted to this area. 
Until basic reading and math skills are mastered, the need and the 
advisability of focusing additional goals on reading, writing, and 
language did not appear to be warranted. 
Long-Term Seals ^ v - 

For the psychological report, long-term goals were defined as those 
goals identified for the^ student 1>l\at may take^moVe than a year to 
master. Long-term goals recommended we're* divided into 15 categories. 

■ 34 



Table 3 (continued) 



EKLC 



• 

f ' 1. M 

Cateqorles 
Findings and Recommendations 
Taken From Testing Reports 

— - 1 


1980 


198] 


1982 


6z 

X 


S II 

X 


S H 
ij 

0 z 
X 


4ir ^ rs 

X 


to fO 

in 

6z 
X 


U GO 

^ II 
H Z 

X 


1 

C/i 00 

3 H 
^ Z 

X 


X 


10 o 
"S 

3 u 
U Z 

X 


JC U 

% 


$ ^ 

2 " 
O Z 

X 


Total I of 
M Rpts Reviewed 

N = 70 


Lonq-Terrt fioals 

Improvement of English Language Skills (Receptive and Expressive, 
Auditory Discrimination) 


4? 


44 


26 


' 40 


57 


69 


38 


62 


65 


61 


38 


^ 54 


Improvement in Academic Skills 


42 


36 


29 


38 


30 


40 


0 


33 


5 


50 


25 


29 


General Improvement of Math Skills 


37 


49 


• 42 


41 


39 


38 


63 


41 


55 


38 


33 


41 


improvement of Specific Hath Skills (Application, To Age or Grade 
Level, Addition^ Multiplication, etc.| 


24 


13 


0 


17 


0 


2 


13 


3 


55 




— 54" 




Herteral Improvement of Reading Skills 


46 


52 


65 


51 


35 


46 


. 75 


46 


50 


46 


54 


50 


Improvement of Specific Reading ^Skl lis (Comprehension, 
Word Attack, Phonics, etc.) 


25 


21 


0 


20 


0 


0 


0 


0 


30 


42 


46 


40 


beneral Improvement of Writing and Spelling Skills 


16 


21 


0 


14 


30 


21 


50 


27 


25 


23 


38 


29 


Improvement of Specific Writing and Spelling Skills (Two and 
ihree Letter Words, Irregular Words, Grammar, etc.) 


14 


13 


0 


11 


0 


0 


0 


0 


0 


15 


4 


7 


Improved Social Skills (Self -Concept, Self-Conf idence. 
Interaction, etc.) 


59 


23 


29 


45 


65 


40 


38 


47 


^ 45 


31 


21 


31 


Improved Behavioral Skills (Appropriate Class Behavior, Attention 
Span, Independence, Completion of Assignments, etc.) 


19 


33 


26 


23 


13 


31 


63 


29 


30 


31 


17 


17 


toordination and Visual-Motor Skills 


13 


8 


19 


13 


4 


22 


0 


8 


15 


19 


0 


11 


Increased Participation (Includes Academic and Nonacademic) 


1 


3 


0 


I 


0 


2 


0 


1 


0 


15 


8 


9 


Attendance Improvement in School 


12 


0 


6 


9 


9 


13 


. 0 


10 


0 


0 


13 


4 


Independent/Survival Livinc| or Work Skills 


9 


10 


0 


8 


9 


•2 


0 


4 


15 


12 


8 


n 


Vocational Skills 


10 


10 


10 


10 


4 


4 


13 


5. 


20 


15^ 


8 


li 


. ^ TOTAL 
" A Wr. ■ * 


.370 


333 


352 


358 


296 


248 


313 


268 


410 


.438 


366 


406 
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Table 3, page 35, indicates that the most common long-term goals 
reconwended included improvement in the use of English language, math 
skills appropriate to the age and grade level of the child, and general - 
improvement in reading. Approximately half of the reports contained 
recomftiendations for long-term instructional goals in math, reading, and 
English language. The next most frequently recomnended long-term goals 
included improvement in social skills, general writing and spelling 
skills, and general improvement in academics. Long-term goals, for the 
most part, were less specific, but their pattern was very similar during 

the course of the study. 

The subtotals category for long-term goals suggest that the average 
psychological report contained from three to four long-term goals. 
Instructional Strategies and Materials 

While testing each child, the psychologists were instructed to make 
notes -of specific behaviors and learning styles of the child. Utilizing 
this information along with sftort- and long-term goals, psychologists 
were asked to identify specific instructional strategies or materials 
which they felt would capitalize on the child's learning strengths. 

Table 3, page 37, identifies six categories of instructional ' 
strategies and materials. Specific instructional techniques, including 
auditory or visual approaches, and drill and practice exercises were 
recommended in approximately 25% of the reports. Behavior modification 
and management programs were recoimended for approximately 30% of the 
cases. These percentages were relatively consistent during the course of 
the study. The specificity of programs and activities named in the 
reports increased throughout the three years of the project from only 6% 
In 1980:to 37% in 1982. This change suggests that psychologists 

• 36 50 . 



Table 3 (continued) 
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Cdte(|ories 
Findlnqs and Recbinmendatlons 
Taken From Testing Reports 


g* -- - 

1980 - 


1981 


1982 « 


m o 

X. rH 
T 11 

6^ 

X 


II 

H Z 

X 


1 

/fl II 
0 ^ 

X 


u-i 1 

°? 

Ik 

' X 


I" 

% 


tJ 00 
43 If 

^ 2 

% 


(A 09 

a ii 
J' 


%\ 

% 


ffl o 

Sc. tN 

3 " 


'0 

4J rsi 
*g U 
H Z 
X 


9 u 


05 o 

D a " 
H a » 


Instructional Strategies and Materials 

Sroall Group Instruction and/or Individual 

Instruction with Student or Peer Tutors 


60 


33 


39 


50 


35 


31 


38 


33 


- 

25 


27 


• 

4 


19 


Increased or Decreased Time on Tasks i 

~ ' " — — — -- . - — — — — — ■t- 


2 


0 


0 


1 


0 


0 


0 


0 


10 


4 


0 


4 


Specific Program or Activity Named i 


.. 8 


• 8 


0 


6 


26 


30 


> 

25 


24 


55 


23 


38 


37 


Title I Placement Materials 




3 


0 


3 


0 


0 


.0 


•0 


0 


0 


- 0 


0 


...... J . _, _ . - , ^ 

Behavior Modification or Management Program * - 
(Reinforcement, Incentives, etc.) 


36 


21 


6 


27 


17 


27 


38 


25 


25 


27 


33 


29 


Specific Instructional Techniques (Visual-Motor^ Auditory 
Approaches. Audio-Visual, Drill and Practice Exercises, 
Flashcards, etc.) 


22 


44 


10 


24 


22 


'31 


13 


27 


10 


31 


4& 


30 


"Ijo Instructional Strategies Named 




0 


0 


8. 


0 


0 


0 


0 


15 


15 


4 


11 


TOTAL 


\ 144 


H)8 


55 


120 


100 


113 


113 


109 


♦ 

140 


127 


125 


130 


Support Services 

Visual Examination' or Treatment 


1 

31 


21 


. 25 


28 


35 


23 


13 


> 

25 


20 


19 


38 


26 
46 


Hearing Examination or Treatment 


36 


38 


29 


35 


30 


38 




11 


30 


65 


38 


Medical Examination or Treatment 


10 


3 


0 


6 


4 


8 


25 


9 


25 


/ "a 


4 


11 


Psychiatric Examination or Treatment 


1 


0 


0 


1 


0 


2 


0 


' 1 




0 


4 


1 


«>pcccn incrdpy ana/or f^ssessmeni 


5 


3 


0 


3 


4 


16 


38 


15 




0 


8 


9 


Counsel in<] ' i 


31 


3 


16 


22 


30 


31 


25 


30 


"? 


19 


29 


31 


Behavior Consultant (For Behavior Modification) 


6 


3 


16 


7 


13 


6 


13 


9 




0 


4 


\ 


• ^ TOTAL 


121 


-69 


6? 


103 


117 


125 


163 


127 


/ 145 


112 


1 

125 


126 
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increased in their specificity of instructional strategies and materials 

which they felt would assist in remediating the child's difficulties. 
♦ 

Support Services 

During testing, psychol<3g1sts are provided an opportunity to observe 
the Chi Id under a variety of conditions: direct instruction, independent 
work, problem solving, pressures of expectations, and time constraints. 
Through such observations, various health, behavior, or psycTiological 
problems can be identified. The Support Services Section on the testing 
report requested the jjsychologist to give recommendations for special 
services such as medical exams, physical therapy, consultation, and 
hearing evaluations that would not normally be provided by the teacher in 
special education. Table 3, page 37, identifies seven categories of 
specific support services recomnended. The most frequently recommended 
support services were for hearing examinations or treatment . Approx- 
imately 35% of the students tested- were recommended as needing additional 
hearing examinations and/or treatment. Counseling was recoimftended in 
approximately 30X of the cases followed by visual examinations. The 
average number of support services recommended on each report was one and 
a half. 

Evaluation Criteria for Success 



The final section of the Psychological Testing Report Format 
requested psychologists. to Identify the criteria needed to evaluate the_/ 
success of the child's educational program/ Seven categories of recom- 
mendations were id'e/itif ied. The most frequently recommended criteria for 
success was improvement in academic skills, with Ippr ox i mate ly 80X of the 
testing reports making this a recommendation. Improved social behavior 



was recommended in approximately 30* of the reports, and improved 
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Table 3 (continued) 





1980 


1981 


1982 


Categories 

Findings and "Recoinmendat ions 
Taken Frofn Testing Reports 
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Total # of 
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• 
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% 


% 


% 


X 




X 




X 


X 


% 


X 


. Evalu^tion^Criteria for Success 


























Improved Academics Skills 


72 


69 


67 


70 


96 


90 


88 


91 


90 




100 


94 


Improved Communication and Language Skills (Receptive, 
Expressive, and Auditory Discrimination) ^ 


cc 


o 
O 


3 


15 


0 


6 


13 


5 


35 


35 


21 


30 


Increased Participation (Academic and Nonacademic) 


5 


3 


0 


4 


fx 

0 


0 


0 


0 


5 


4 


8 


6 


Improved Soci<lV Skills (Interaction, SeH-Concept, 
Self-Conf idence, etc.) 


40 


10 


6 


27 


17 


V 

6 


0 


9 


45 


31 


29 


34 


Improved Behavioral Skills (Appropriate Behavior, Attention 
Span, Independent Class Work. Responsibility, and 
Completion of Assignments 


31 


26 


39 


31 


52 


35 


50 


42 


20 


27* 


33 


27 


Vocational and Independent Livinf) Skills 


8 


8 


10 




0 


4 


0 


3 


10 


19 


8 


11 


Improved Visual-Motor Coordination Skills 


17 


5 


6 




r 0 


^ 8 


0 


5 


10^ 


4 


, 0 


•-^ 4 


TOTAL 


195 


141 


132:; 
— - — * 


122 


70 


150 


150 


127 


215 1 


212 


200 


-tr-7 

'209 



■I 



55 



communications and language skills were recommended in approximately 20% 
of the reports. Improved independent living and vocational skills, motor 
coordination, and self-concept were recommended to a lesser extent. This 
finding may reflect the age of the children tested and the fact that most 
of the deficits observed on the children were in the early academic areas 
of reading, math, and language. 

The second and third method of collecting data to determine Objective 1, 
i.e., the strengths and weakness.es of the psycho-educational assessment 
report system and process for the Fort Defiance Agency, included: 

1. A critique of a sample of test reports utilizing the Test Report Critique 
Form by a sample of school personnel from the target schools. 

2. A structured interview in which comnents, suggestions, recommendations 
and criticisms were solicited. 

Since these data served as the baseline to determine the impact of the 
study, these data will be reported along with changes that occurred when the 
same data were collected at midpoint and at the end of the project in 
Objective 3, Tables ,4 through 14. 

Objective 2 - Based on the Assessment Oat a Collected In Objective 1, An 
Instructional P^^ogram Win Be Designed to Train: (A) Teacnlng and 
Administrative scnooi Personnel, and (B) School Psychologists 

The second objective of this study was to provide inserv ice training 

designed to improve the usefulness and utilization of individual testing 

reports by the school staff in the Fort Defiance Agency and the psychologists 

who provided testing to the agency. To accot^plish this objective, two 

workshops and two seminars were planned. . 

t m 
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1. Inservice Workshop I 

The first inservite workshop and seminar were provided during the 
suimier of 1981 (school personnel and psychologists). The second' wdrkshop 
and seminar were provided to similar groups of people during the summer 
of 1982. The original proposal indicated that the content of the 
inservice training workshop and seminar for both school personnel and ECC 
psychologists was to be identified from the dala collected for 
Objective 1. Following the analysis of the 1980 testing reports, ^everal 
specific needs were identified by the principal investigator and the 
director of special education that were felt would improve the relevance 
and utilization of psychological testing reports by the Fort Defiance 
school personnel. As a result of this analysis, the first inservice 
training workshop was designed and presented August 11-14, 1981, to the 
school personnel of the Fort Defiance Agency. Topics addressed in 
Workshop I included the following: 

a. Statement of the Referral Question : The data collected and analyzed 
from Objective 1 emphasized the relationship between a well stated 

r 

referral question and the usefulnes of the resulting testing report. 
For the most part, the specificity of answers in the reports 

> 

corresponded closely to the specificity of questions asked by school 
personnel. When referral questions were vague and general, findings 
and recommendations 1n the testing reports were rated by school 
personnel as vague and general. On the other hand, when referral 
questions were specific, these data suggest that school personnel 

needed information on methods of identifying and stating referral 

I 

questions. The workshop presented information on the importance of 
the referral question, examples of good and bad referral questions, 

1 
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along with a series of exercises in which participants prepared and 
critiqued referral questions (see Resource Manual, pages 41-43). ^ 

b. .Location and Access to Testing Reports : Table 4, page 43, indicates 

that 40* of the school personnel interviewed jn 1981 had never 
referred to a testing report and did not know that they were avail- 
able. In addition, 20X indicated that they did not refer to the 
testing report because the information they needed was provided by 
another school staff member (school psychologist, supervisor, 
administrator or the director of special education). 

In response to these findings, information concerning the 
location of the testing data in the school, v^o had access to it, and 
how it was to be used was presented in the workshop (see Resource 

% 

Manual, pages 212-213). 

c. Bias in Testing ; • Verbal comments from school personnel expressed 
concern about using standardized tests. They emphasized the lack of 
confidence school personnel had in standardized tests. The school 
personnel interviewed were particularly sensitive to bias in testing 
due to cultural or language factors. Their comments suggested they 
were overly concerned and overemphasized the limitations and problems 
of standardized tests. Accompanyirjg this was evidence that school 
personnel were unaware of the great efforts to reduce bias in 
Instruments and procedures and attempts to use other assessment 
options rather than standardized tests.' The workshop provided 
information to school personnel on efforts to minimize bias in 
testing both procedurally and in the selection of instruments, and 
demonstrations of nonbiased testing along with the use of other 
assessment insitruments (see Resource Manual, page 3). 

7 • ^ 



Table 4 



The Extent School Personnel Used Psycho-Educational 
Testtnq Reports During the School Year 





Chuska 


Tohatchi 


Greasewood 




Total 








Extent To Wl>lch Testing 
Reports Were Used 


N - 12 

V 


1981 
N = 13 


7 


1980 

N - 5 
X 


1981 
N - 16 


1982 
N " 8 


T?80^ 
N = 4 


N - 9 


"Si 


1 


, 1980 

h.n 


1981 
N - 38" 
X 


1982 
N ■» 20 
'% 


■ T 
80-82 


Of 


Never Used, Didn't Know 
They Were Aval 1 able 


0 


0 


0 


40 


6 


13 


75 


33 


40 




24 


11 


15 


.73 


39 


Never Used, Needed Information 
Came From Consultation 


58 


23 


0 


20 


13 


13, 


0 


44 


20 


< 


38 


24 


10 


2.23* 


39 


Used Once at lEP Meeting 


17 


31 


43 


20 


38 


38 


25 


ir 


40 




19 


29 


40 


1.51 


39 


Used Otten for Irl' 
Oevelopmeiit 


17 


23 


43 


20 


31 


23 


a 


11 


0 






24 


25 


.89 


39 


Used Several Times 
During a Year 


3 


23 


14 , 


0 
r 


12 


13 


0 


0 


0 




5 


12 

X— . 


10 


: .61 


39 



*£<.05 
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Using Test Reports in Writing lEPs ; Table 4 indicated that in 1980, 
only ]9% of the school personnel reported they used testing reports 
in the lEP meeting. It was felt that this low rate of using testing 
reports for developing lEPs was partially due to school personnel not 
knowing how to *stract and utilize testing report data in the 
development of lEPs. r 

To address this issue, a series, of simulation exercises were 
designed, in which testing reports were provided and participants were 
asked to develop the lEP from the testing report da*a (see Resource 
Manual, q^ges 101 through 115). 

Other Topics : In addition to the other four topics, the workshop 
agenda included video-tape presentations with accompanying work 
activities on the role of the regular teacher in special education, 
P. L. . 94-142, and the philosophy and rationale for* individual 
• assessment. 
Evaluation of the Inservice Workshop 

At the completion of each component of the workshop, partici- ^ 
pants were asked to evaluate it. The evaluation is contained on 
pages 129 through 140 of the Resource Manual. The results of the 
worksho-p evaluation, together witrt informal comments, criticisms-, and 
recorrmendatioas were collected and analyzed. 

The evaluation results suggest that the first workshop was well 
received and participants felt it was interesting and relevant. 
Logistical problems such as timing, location and equipment effected 
the comfort of participants. A careful record was maintained by the 
workshop director and each presenter, listing suggestions for 

44 
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conducting filfrure workshops. These suggestions were implemented in 
the 1982 inservice training workshops. 
Seminar I 

The first seminar was designed for the psychologists from the 
Exceptional Child Center- selected to provide psychological testing for 
the Fort Defi ance Agency in the fall of 1981. This seminar was conducted 
August 27, 1981 with primary emphasis focusing on: (a) obtaining and 
interpreting referral information, background information from student 
records, interview^ing teachers and Navajo children, (b) interpretation of 
resource material including the STEP Program, the Brigance and other 
instructional materials widely used by the Fort Defiance Agency, 
(c) correlating test findings and reftemmendations with specific goals and 
subgoals in the STEP Program, and (d) how and when to utilize a Navajo 
interpreter. ^ 

The above topics were identified by the principal investigator and 
the director of special education in the Fort Defiance Agency by analyz- 
ing data and informal comments of school personnel interviewed (see 
Resource Manual, pages 254 through 266)l.'' 

4 

Inservice Workshop II 

The second workshop was designed for school personnel of the Fort 
Defiance Agency and presented August 11-J2, 1982. The content of this 
inservice workshop was similar to that of the first workshop. It again 
focused on the r^faj^al question, bias in test1>ig, and utilizing testing 
results in writing ifePs. However, it was not felt necessary to include 
in the agenda a discussion of the location and accessibility of testing 
reports, etc. Thus, extra topics were/'added that included Public Law 
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94-142 and instructional evaluation (precision teaching) (see Resource 
Manual, pages 142 through 233). ' 

to evaluate the second workshop, a pre- and posttest was developed. 
The results of the workshop evaluation are presented in the Resource 
Manual , pages 234 through 248. 
4. Seminar II 

^The second seminar was provided to the school psychologists selected 
to provide psychological services to the Fort Defiance Agency in the fall 
of 1982. This seminar was conducted on August 28, 1982. 'The content of 
the seminar was similar \o the first seminar provided to school psychol- 
ogists in 1981. The most significant' emphasis of the 1982 seminar was on 
^ writing and editing testing reports. Simulated exercises were provided 
for writing testing reports and conducting meaningful "high-speed 
staffings" to report test results and obtain additional information from 
school personnel. The agenda for the second' school psychologists 
seminar, exercises, resource material, etc. is provided in the Resource 
Manual, pages 279* through 32T. 

Objective 3 - As a Result of the Training Conducted in Objective 2. Testing 
Results Win Be Pound to Be Mortf valuable in Making Decisions Concerning the 
Placement of Native American students in special education Programs 

The third and final object1ve'^^<jfJ|)/rs project was to determine the 

impact the project made in improving the utilization and educational 

relevance of individual psycho-educational assessment 1n placing and develo'p- 

> 

ing lEPs for handicapped Native American children. The original grant 

proposal identified four broad criteria by which the impact df'vtbfiLJnterv.en- 
l 

tion activities, i.e,' change in the individual testing process and testing 
O 46, . 
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format, aiyl inservice training provided school personnel and ECC psychol- 
oqists, of the project could be determined: 

1. As a result of the intervention activities, the testing results wil.l be 

used more frequently by school personnel. 

« 

2. As a result of the intervention activities, testing reports will be 
perceived by school personnel as being more usable and understandable. 

3. ' At the completion of the project, there will be a greater relationship 
between the instructional program proposef*- in the lEPs of students and 
those recoimended on the psycho-educational testing reports. 

4. At the completion of the project, referral forms, testing reports, and 
♦ other procedures used to communicate individual test findings and 

recommendations will have been improved. 

To determine the impact of t]ie project, baseline data was collected in 
March of 1981. These same data>were collected once again in February 1982 
and again at the end of the project in November 1982. The differences 
betwee^n the data collected at the beginning of the project, at midpoint, and 
at the conclusion of the project were compared to determine the impact of the 
intervention activities initiated by the project. 

By utilizing three observations, trends in the data could be more 
clearly determined and appropriate changes could be initiated during the 

* 4 

project' to improve the usability and relevance of individual assessment. 

This section will address the data collected and findings which address 

the four criteria selected to determine the impact of the study. 

1 . As a result of the intervention a(;tivities, the testing results will be 
used more frequently by school pei^sonnel 

Two techniques were proposed to determine the extent psychological 

testing reports were used by school personnel: (a) counting and then 
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comparing the number of times School personnel checked out testing 
reports from the building testing file, and (b)' asking school personnel 
utilizing a questionnaire and a fo'llow-up interview. 

Counting and recording the number of times school personnel checked 
out testing reports proved to be more difficult than originally Antici- 
pated because more than one copy of the testing report was available in 
the district. Often the testing report used came from the district 
office and there was no need to check out^the building copy. In addi- 
tion, subsections of reports were sometimes copied and summarized and 
utilized in lEP' meetings. Furthermore, there was no consistent way of 
monitoring whether people actually did sign out before using testing 
reports placed in the building testing file. 

As ^ result, data to determine the extent and the purpose for which 
psychological testing reports were used during the course of the study 
were obtained by utilizing an interview and a questionnaire. These data 
are presented on Table 4, page 43, and Table 5, page 49. Table 4 
indicates that in 1980, 24% of the staff interviewed had never used • 
testing reports and did not, know that they were available or that they 
had access to them. Ah additional 38X indicated that they never referred 
to psychological testing reports because the information they needed came 

from consultation with other people in the agency (director of special 

. ■ \ 

education, principal, school psychologists, counselor's). Of those school 
personnel reported they did use psychological testing reports, 19X 
reported that they had used them once at the lEP meeting, 14X used them 
often for lEP development, and only 5% used the reports several times 
during the year. , . 




Table 5 

Ihe^^urpose School Personnel Refer to 
Psycho-Educational Testing Reports 



J- 



rurpose Keporus were usea 


Chuska 


Tohatchi 


.Gfeasewood 


1 r- "' ■ ' " 

.Total ; 






\m 

N - 12 
X 


N - 13 
% . 


1982 
N - 7 
% 


1980 
N » 5 
% 


1981 
N - 15 
X 


3982 
N - 8 
X 


1980 
N » 4 
X 


^ 1981 
N - 9 
X 


1982 
N - 5 
X 


T980 
N - 2i 
X 


1981 
N - 38 
X , 


1982 
H - 20 
X 


t 

80-82 


DF 


Did Not Refer to the Test Reports 


SO 


38 


-0 


60 

• 


19 


37 


,75 


33 


^0 


^ 57 


29 




2.2J* 


39 


To Determine Eligibility 

for Special Education 


50 


62 


100 


40 


81 




25. 


67 


'60 


43 


71 


75 


2.21* 


39 


To Obtain General Information ' 
on Ability and Achievement 


17 


4 


14 


'20 


50 


25 


0 


22 


20 




34 


20' 


.sr 


. 39 


To Help Identify Problems 
and Causes 


8 


23 


51 


20 


19 


13 


25 


n 


20 


14 


18 


30 


1.26 


39 


■ - ■ — f— 

To Kind Recommendations ■ 
for Planning 


8 


IS 


29 


0 


13 


13 


0 


11 


' 11 

t 


5 


11 


« 

20 


1.48 


39 



0 



t9k 



*£<.05 
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During the course of the study, the extent that school personnel 
used testing reports increased. Ip 1982, the number of people who 
reported they never -used testing reports and obtained needed testing 
information from some other source reduced significantly. 

Table 5 presents data on the purpose, for which school personnel used 

'Psychb-educattona>^esting reports. These data indicate that the 

percentage of school personnel who reported that they tl id not use the. 

testing reports reduced significantly during the course of the study and 

the percentage of school personnel who reported using testing reports to 

determine eligibility for special education increased significantly. 

Although not statistically significant, the data indicate a trend for 

school personnel' to incr*ease their utilization of testing reports to 

obtain information on student «:hievement and ability, and to help 

identify problems and to find recommendations for planning. This 

suggests that at the conclusion of the study, teachers were using testing 

reports for more sophisticated purposes. » 

As a Result -of the Intervention Activities, Jesting Reports Will Be 
Perceived by School Personnel as. Being More Usable and Understand- 
able '. — - I 

Data to determine if the project resulted in testing reports 

* *■ .. / • 

1 

perceived by school personnel as beiaikmore usable and .understandable 

were collected by using the Psycho-EdSSkional Testing Report Critique 

Form. The Testing Report tritique Form contained^ .items to evaluate the 

following five subobjectlves. * '111 ' 

a. To What Extent School Personnel Felt 'Individual Sections of the . 
Test^g Report Met Their Information Needs 

• Table 6,« page 51, provides a summary of how school personnel 
' ♦ • - 

felt the specific sections .of the psycho- educational testing reports 
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r Table 6 ' ^ 

School Personnel Evaluation of the Separate Sections of the Psycho-Educational Reports 

"A" 

The Extent School Personnel Felt Excerpts of Specific Sections of the Report Met Their Information Needs 



< 

* * 


VERY WELL, Every 
Objective Was Met 


good'. Most of the 
Objectives Were Met 


FAIR, A Few of the 
Objectives Were Met 


POOR, Almost None 
of the Objectives 
Were Met 


No Respon 


se 


N - 21 
% 


1981 
N - 3fi 
% 


15/12 
N - 2( 

% 


1380 
N = 21 
. % 


1981 

N » 3fi 
% 


1982 

N » 2C 
% 


1980 
N « 21 

0/ 


1981 
N » 3f 
% 


1982 
N - 2C 
% 


1980 
N » 21 
% 


^gBT 

N - 3t 
X 


1982 
N - 2(1 
% 


1980 
N « 2^ 
% 


1981 
N - 3^ 
1, 


1982 
N » 2( 
1, 


Referral Informa'tion 

1980 1981 1982 
Chuska ' N»12 N»13 N-7 


-r 


38 


43 


42 


38 


29 


fi 


23 


29 


0 


0 


0 


0 


0 


0 


Tohatchi N>5 N'16 N«fi 




31 


75 


60 


44 


25 


20 


19 


0 


0 


, 6 


0 


0 


0 


0 


Grcasewood N^Q 


25 


39 


40 


25 


. 55 


60 


0 


11 


0 


25 


' 0 


0 


25 


0 


0 ' 


Total • N - ?i M » 3fl N - 2ff ■ 


35 




-55 


43 


45 


35 


' 10 


— rr- 


■ TO' 


5 


3 


0 ■ 


5" 


0 


■ -n 


Background information * ^ 

1980 1981 1982 
Chuska /--^ = 12 .N ='l3 N = 7 


42 


7 


43 


25 


69 


57 


17 


15 


0 


17 


8 


0 


0 


0 


0 


Tohatchi ( N»5 N»16 N»8 


40" 


25 


75 


60 


. 44 


25 


0 


31 


0 


0 


0 


0 


0 


0 


0 


^>reasewood V^l=4 N=9 N=5 


5 


33 


40 


25 


33 


40 ■ 


0 


33 




0 


0 


0 


25 ' 


0 


0 


Total '^^ ^ N = 21 k ^Ji ' W » 










- 5ff ■ 


40 


10 


26 


■? 


— Ttr 


3 


■ .0 


5 


0 


O" 


Behavioral Observations 

1981 ^ 1982 
Chuska N * 12' N = 13 N = 7' 


67 


77 


57 


Ifi 


23 


43 


- 

8 


0 


0 


0 


• 

0 


0 


8 


0 


0 


Tohatchi N=^5 N=^16 N=8 


100 


50 


75*' 


0 


38 


■25 ■ 


0 


6 


0 


■ 0 


0 


0 


0 


— cr 


0 


TfnMsewood N=4 N=9 N=5 


I 50 


55 


SO 


25 


44 


20 


0 


0 


0 


0 


0 


0 




0 


0 


Total - N = Jl * 3fl f< - 20 




6T' 




■■ 14 


34 


30 


5 


- 3 


0 


0 


0 


0 


10 


0 


0 


Psycho-Educational Findings 

1980 1981 1982 
Chuska N''12 N°13 N»7 


« 
f 

*75 


f- 

77 




8 


23 


43 


' 17 


0 


0 


0 


0 


» 

• 0 


0 


0 


0 


Tohatchi N 5 N ^ 16 ' N > d 


100 


65 


00 


0 


44 


12 


0 


0 


0 


0 


0 


0 


0 


0 


0 


Creasewood N^4 N^9 N'5 


50 " 


, 55 


60 


0 


44 


40 


25 


0 


0 


0 


0 


0 


25 


0 


0 


Total N = ?! 38 ■ M = 2fl 


"~7^ ■ 


—m 


70 


5^ 


- 37" 


30 


— TT" 


0 ' 




' 0" 


— rr 


— 0- 




0 


0 


Sunimary 

1980 1981 1982 
Chuska N='12 N»13 N«7 


58 


62 


66 


33 


62 


14 


8 


0 


0 


0 


0 


0 


0 


8 


0 


Tohatrhi N^ll N«l6 N'8 


fiO 


3S 


100 


20 


63 


0 


0 


0 


0" 






0 


" 0 


0 




6reasewood N-4 N»9'N>5 


25" 




flO 


50 


33 


20 


0 


0 


0 


•0 


0 




25 


0 


— JP 


Tola! . , N - 21 M - 3A N - 20 






—90- 


33 


— fr- 


■ Tff 






IT" 








5 


3 


0 


RecoMinendat ions 

1980 1981 1982 
Chuska N = 12 N '« 13 N '7 


66 




71 


- 

16 


31 


14 


16 




14 


0 


0 


0 


0 


0 


0 . 


Tohatchi N » 5 N - *16 N - 8 


100" 




100 


0 


44 


0 


0 


* 13 


0 


— ir 


0 


. 0 


0 


B 




Greasewood ^J«4 N-9 N»5 


25 


77 


100 


jp 


22 


0 


25 


0 




. 25 


0 


0 


25 


0 


■S 


Tol^T"^ ' N . 21 N - 3fl M . ?fl 










32 





-IT" 


R 


— r 


— T" 




1 — g- 


5 


— r 




Diaqrvostlc Statement 

1980 19fH 1982 
Chuska N = 12 N 13 N » 7 ^ 




w 69 


43 


R 


23 


43 


25 


0 


14 


0 


8 


0 


0 


0 


0 


Tohatch'i N - 5 N - 16 N - 0 




50 


ion 


20 


44 


0 


0 


6 


— ir 


0 


0 


0 


0 


0 


0 


kGreasewood N»4 N«9 M«S 


. 0 


55 


80 




44 


20 


25 


0 


^ 0 


0 


b 


— (T 




0 


0 


Total N « 21 N ■ 38 N - 20 


S7 


58 




-J- 


ir 


20'- 




T'- 


5- 


— ir 








"■"■ir 
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met their information needs. Responses to questions concerning the 

seven subsections of the psychoXeducational testing report for 1980 ' 
r 

indicate that approximately 58% were in the most favorable category. 
The value of referral information vyas rated in the most favorab>e 
category by only 38X of respondents. The most favorable subsection 
of the testing reports as evidenced by ratings in the most favorable 
category was the psycho-educational testing findings in which 76% 
felt this section met their information needs very well. 

The percentage of responses in the most favorable category 
increased during the 'course of the study for subsections of the 
reports addressing referral information, background information, 
summary, recommendations, and the diagnostic statement. There was 
little change in the percentage of rJ^onses in the most favorable* 
category on behavioral observations, and a slight decrease in the 
^percentage of respondents in the most favorable category on psycho- 
educational findings. Table 6 further indicates a reduction in the 
percentage of people who responded in the least favorable categories 
during the course of the study and the number of no responses reduced 
significantly. 

To determine if the change that occurred in the ratings of 
reviewers between 1980 and .1982 was significant, the difference 
between independent sample proportions" t test was used. 




Tab^s 6a, 6b, and 6c indicate rthat the more favorable ratings 
of reviewers between 1980 and 1982 of the referral, background, and 
behavioral information sections on the tests were not significant at 
the .05 level. A general trend for school personnel to evaluate 

these sections in the more favorable category as the project contin- 

c 

ued was, however, noted. Concomitant with this -was a decrease in the. 

»• 

percentage of reviewers who rated the prementioned sections as ei.ther 
fair or poor. 



Table 6a 

The Extent School Personnel Felt the Referral 
InforTnation In the Reports Met Their Information Needs 



1980 1981 1982 t 
Response . X . % . % . 1980^82 . OF 



Very Wei 1, Every 
Objective Was Met 


38 


34 


ft 

55 


1.11 


39 


Good, Most Objectives 
Were Met 


— T — " 

A 

42 


45 


35 


.46 


39 


Fair, A Few of the 
Objectives Were Mtft 


10 


18 


10 


0 


'39 


Poor, Almost None of the 
Objectives Were Met 


5 ' 


3 


0 


1.05 


39 


No Response 


5 


0 


0 


1.05 


39 



£<.05 



* 

r 
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Table 6b 

The Extent School Personnel Felt the Background 
Infomatlon In. the Reports Met Their Information Needs 



1980 



1981 



1982 



Very Wei 1 , Every 
Objective Was Met 


43 


21 


55 


.77 


39 


Good, Most Objectives 
Were Met 


32 


50 


40 


.53 


39 


Fair, A Few of the 
Objectives Were Met 


10 


26 


5 


.61 


39 


Poor, Almost None of the 
Objectives Were Met 


10 


3 


0 


1.53 


39 


No Response 


5 


0 


0 


1.05 


39 ' 



£<.05 



Table 6c 

The Extent School Personnel Felt the Behavioral 
Observations In the Reports Met Their Information Needs 



1980 



1981 



1982 



Very Wei 1, Every ' 
Objective Was Met 


71 


62 


70 


.07,, 


39 . 


Good, Most Objectives 
Were Met ^ 


14 


35 


30 


. 1.26 


39 


Fair, A Few of the 
Objectives Were Met 


- 5 


3 


0 


"1.05 


. 39 


Poor,' Almost None of the 
Objectives Were Met 


0 


0 


0 


0 




No Response 


10 


0 


0 


■ 1.53 


39 



£<.05 
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The extent school personnel felt that the psycho-educational 
findings section of the reports met their information needs 
throughout the course of the study did not present a clear trend. 
The percentage rating this section in the most favorable category 
decreased while the second (host favorable category (Good, Most 
Objectives Were Met) showed a significant positive change. This 
change resulted primari'ly from a reduction in 1982 of the responses 
rating the psycho-educational findings section as fair or poor. 



I 



able 6d 



The Extent School Personnel Felt the Psycho-Educational 
Findings in the Reports Met Their Information Needs 



Response 



1980 
% 



1981 
% 



1982 
% 



1980-82 



Very Well, Every 
Objective Was Met 


76 


62 


70 • 


.43 


39 


Good, Most Objectives 
Were Met 


5 


38 


30 


2.21* 


3-9 


Fair, A Few of the 
Objectives Were Met 


14 . 


0 


0 


1.85 


39 


Poor, Almost None of the 
Objectives Were Met 


0 


0 


0 


0 




— rz V ^ 

No Response 


' -5 


^ 

0 


0 


1.05 


39 • 



OF 



^p<.05 



Table 6e indicates that significant improvement occurred, in 
the opinion of reviewers, of the sumnary section ^of the psychological 
testing reports. Significantly higher ratings were accompanied by 
reductions in the less favorable categories during the dourse of the 
study. - 
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Table 6e 



The Extent School Personnel Felt the Sumnary 
Section 1n the Reports Met Their Infomatlon Needs 



1980 1981 1982 t 
Response % % % 1980-82 DF 



Very Wei 1 , Every 
Objectives Was Met 


57 


53 


r 

90 


2.60* 


39 


Good, Most Objectives 
Were Met 


33 


44 


10 


1.88 


39 


Fair, A Few of the 
Objectives Were Met 


5 


0 


0 


1.05 


39 


Poor, Almost fjlone of the 
Objectives Were Met 


0 


0 


0 


0 




No Response 


5 


3 


0 


1.05 


39 



*£<.05 ' \ 



Although .it appeared that the recommendations section of the 
testing repfcrts were rated as more favorable at the conclusion than 
at the beginning of the stiNv, this change was not statistically 
signif icant.\ The expected trend towards improvement in the recommen- 
dations section are somewhat conXounded by the 1981 data. 
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Table 6f 

* The Extent School Personnel Felt the Recommendations 
/pfn^the Reports Met Their Information Needs 

1980 1981 1982 t 
Response / X % % 1980-82 



DF 



Very Wei 1 , Every 
Objectives Was Met 


67 


58 


90 


1.86 


39 


Good, Most Objectives 
Were Met 


.10 


32 


5 


.61 


39 


' Fair, A Few of the 
Objectives Were Met 


14 


8 


5 


1.0 


39 


Poor, Almost None of the 
Objectives Were Met 


5 


0 


0 


1.05 


39 


No Response 


5 


0 


0 


• 

1.05 


39 



£<.05 



Vhe. data on Table 6g suggest that school personnel felt the 
diagnostic statement section of the testing reports improved, but the 
percentage of reviewers who rated this section more positively was • 
not significant. 



ERIC 



57 



Table 6g 

The Extent School Personnel Felt the Diagnostic 
Statement in the Reports Met Their Information Needs 



1980 



1981 



1982 



^ 

Very We i i , every 
Objective Was Met 


57 


58 


75 


1.24 


39 


Good, Most Objectives 
Were Met 


19 


36 


20 


.08 


39 , 


Fair, A Few of the 
Objectives Were Met 


19 


' 3 


5 




39 


Poor, Almost None of the 
Objectives Were Met, 


0 


3 


0 


0 




No Response 


5 


0 


0 


1.05 


39 



£<.05 
b. 



The Extent School Personnel Felt Excerpts From Spediffc Sections of 
the Reports Were Useful in Preparing Students' Educational 
Programs" 

To identify the sections that were specifically useful and for 
what purpose they were useful, school personnel evaluated not only 
how well specific sections of the testing reports met their needs, 

\ 

but aflso how useful these sections were in preparing the students' 
lEPs. Table 7 . presents a summary of these dat^ • v 

Table 7 indicates that- in 1980 approximately 655t of the respond- 
ents felt the information contained in the various sections of the 
psychological reports was "very useful"- in preparing students' lEPs. 
In 1982, approximately 85% of the respondents rated the various 
sections of the testing reports as very useful in preparing lEPs. 
Table 7 also indicates a sub,stantia1 reduction in- the number of 
reviewers who did not make a response to this series of questions. 



ERIC 



58 



77 



The 



( Table 7 

School Personnel Evaluation of the Separate Sections of the Psycho-Educational Reports 

"B" 

^xtent School Personnel Felt Excerpts of Specific Sections of the Reports Were Useful in Preparing Students' Educational Programs 





VERY ii: 
A Gi 


>£FUL, It Helped 
-eat Deal and 
iS Necessary 


SOMEWHAT USEFUL, But 
Was Not Really 
Necessary 


NOT USEFUL, And 
Was Not Necessary 


No Response / 


\m 

N « 21 
t 


1981 ' 
N « 38 
% 


1982 
N = 20 
% 


1980 
N » 21 
X 


1981 
N-38 
X 


N « 20 

% 


1980 
N * 21 
% 


1991 
N » 38 
% 


1982 

N - 20 


1980 
H « 21 
X 


1981 
N « 38 

% 


1982/ 
H 20 


Referral Information 

1980 1981 1982 
Chuska N='12 N-13 N=7 


58 


61 


57 


42 


23 . 


43 


0 


8 


0 


0 


8 


0 


Tohatchl N » 5 N »= 16 fJ - fl 


80 


56 


" 88 


20 


38 " 


12 


0 


0 


0 


0 


6 


0 


Greasewood M«4 N«9 M»S 


25 


67 


■ 100 


25 


22 


0 


25 


n 


0 


25 


11 


0 


Tdtal N - 21 N " 38 N » 20 ' 


57 


60 


80 


33 


30 


20 


"5. 


3 


0 


5 


8 


0- 


Background Information 

1980 1981 1982 
Chuska N=12 N=13 N=7 


67 


6? 


57 


25 


30 


43 


8 


- 

8 


0 


0 


0 


■ ■ 

0 


Tohatchi N = 5 N = 16 N - fl " 


60 


44 


88 


20 


44 


12 


0 


6 


0 


0 


6 


0 


Tireasewood N=4 N=9 N=5 


50 


33 


" 100 


0' 


55 


0 


25 


0 


0 


25 


11 


0 


Total N - 21 N n 3fl fj . 


■ 67 


47 


"SO" 


15 


42 


20 


10 




0 


5 


5 


0 


Behavioral Observations 

1980 1981 1982 
Chui^ka N"=l? N=13 N«7 


58 


77 


71 


33 


a 


29 


0 


8 


0 


8 


8 


0 


matclii ■ N = 5 N = 16 N = fl 


100 


62 


■ 100 


(1 


2F " 


0 


0 


0 


0 


0 


12 


0 


Greasewood N=4 N=9 N=B 


50 


77 


. 60 


0 


11 


40 


25 


0 


0 


25 


11 


0 


Total N = 21 N = 3fl N = 20 


67 


>1 


80 


19 


16 


20 


5 


3 


0 


■ 10 


10 


0 


Psycho-Educational Findings 

1980 il981 1982 
Chuska N = 12 N = 13 N »= 7 


58 


77 


r 

86 i 


33 


15 


14 


0 


8 


0 


8 


0 


t 

0 


Tohatch/i ' N - 5 N * 16 N - fl 


100 


81 


100 i 


0 1 


12 


0 


0 


0 


0 


0 


6 


0 


Creasewood N=4 N=9 N=5 


■ 25 


77 


100 1 


0 , 


11 


0 


50 


0 


0 


25 




0 


Total N = 21 N « 33 H = 20 


62 


78 ' 




19 ^ 


16 


5 




3 


0 


TO 


^5 


0 


Stimniary 

1980 1981 1982 
Chuska N = 12 N = 13 N » 7 


58 


62 


I- 
j 

100 


41 


23 


0 


0 


8 


0 


0 


8 


0 


Tohatchi N^S N-16 N«fl 


100 ' 


75 


100 


0 


^ 19 


0 


0\ 


0 


0 




6- . 


0 


Greasewood N»4 N«9 H^S 


■■■■"25 ■ 


flfi 


100 


25 


0 


0 


25 


0 


0 


25 


11 


0 


Total ■ N - 21 N » 3fl H - 20 


62 


74 


100 


29 


l6 


0 


5 


3 


0 


5 


8 


0 


Recoiiinenddttons 

1980 1981 1982 
Chuska N=12 N«13 N«7 


66 


77 


86 


33 


8 


0 


0 


8 


0 


0 


8 


14 


Tohatchi N=5 N=li6 N«8 - 


100 


76 


100 


0 


19 


0 


0 


0 


0 


0 - 


6 


0 


Greasewood N«4 N'*9 fl«S 


■25 ' 


88 


' '80- 


0 


n 


0 


50 


0 


20 


25 


0 


0 


Tntal "TTTTT H » Ifi: — u - 

lOLiii n"ti w" Jo n ■ c\i 


" n • 

0/ 


75 


90 


19 


13 


0 




3 


5 


5 




'5 


Oiaqnostic Statement 

1980 1981 1982 
Chuska N'12 N'13 N»7 


75 


69 


57 


25 


15 


29 


0 


8 


0 


0 


8 


14 


Tohatchi N = 5 N = 16 - N = fl 


100 


81 


100 


0 


13 


0 


0 


0 


0 


0 


— r~ 


jp- 


Greasewood N»4 N«9, N-5 


25 


66 


100 


25 


11 


, 0 


25 


0 


0 






0 


Total N » 21 N-38 N - ZO 


-7T— 


74 


85 


19 


T3 


'10 


'5 


3 




5 


11 


5 



ERIC 



The changes between the 1980 and the 1982 evaluations of the 

•r 

referral section, background information, and behavioral observations 
show a trend towards more favorable categories by a reduction in tW 
percentage of respondents rating these sections as not useful (see 
Tables 7a, 7b, and 7c) . ' 



Table 7a 



The Extent School 
in the 




nnel Felt^thei^R^ferral Information 
Was Useful In Pfeparing lEPs 



1980 



1981 



1982 



Very Useful, It Helped 
A Great Deal and Was 
Necessary 


57 


60 


80 


1.64 


— / ■ • ' 

39 


Somewhat Useful, But Not 
Real ly Necessary 


• 33 


30 


20 


.96 


39 


Not Useful, and Was Not 
Necessary 


5 


3 


0 


1.05 


39 ' 


No Response 


5 


y 

1 


0 ' 


1.05 


39 



£<.05 



Table 7b 

The Extent School Personnel Felt the Background Information 
in the l^eports Was Useful In Preparing lEPs 



1980 



1581 



1982 



■ » ■ ■ ■ 

Very Useful, It Helped 
A Great* Deal and Was 
Necessary 


67 


48 


80 


.96 


39 


Somewhat^Us^^ul , But Not 
Really Necessary 


18 


42 


20 


.16 


39 


Not Useful, and Was Not 
Nefesisftry 


10 


• 

5 


0 


1.53 


• 

39 


No Response 


5 


5 


0 


1.05 


^9 



p<.05 



60 



table 7c * 

The Extent School Personnel Felt the Behavioral Observations 
in the Reports Were Useful In Preparing lEPs. 



1980 



1981 



19'82 



Very Useful, It Helped ' 
A Great Deal and Was 
Necessary 


. 67 


71 


* 

80 


.96 


39 . ' 


Somewhat Useful, But Not 
Really Necessary 


18 


16 


20 


.1.6 


39 


Not Useful, and Was Not 
Necessary 


- 5 


3 


0 


1.05 


39 


No Response 


10 


10 


0 


1.53 


39 



£<.05 



The psycho-educational findings section of the testing reports 
was evaluated as signifTcantly more useful in preparing students' 
lEPs in 1982 than in 1980 (see Table 7d). 

Table 7d 

The Extent School Personnel Felt the Psycho-Educational 
Findings In the Reports Were Useful In Preparing lEPs 



Response . 



1980 1981 1982 ' t 
% % . % 1980-82 



OF 



Very Useful', It Helped 
A Great Deal and Was 
Necessary 


. 62 


77 


95 


2.83* 


39 


Somewhat Useful, But Not 
Really Necessary 


18 


15 


5 


1.34 


39 


Not Useful, and Was Not 
Necessary 


10 


3 


•0 


1.53 


39 


No Response 


10 


5 


0 


1.53 


39 



*£<.05 



61 



8i 



Table 7e shows a very significant change in how useful school 
personnel felt the surmiary section of the reports was in preparing 

lEPs. These data indicate that the summary section As considered to 

V * . 

be very useful, helped a gre^at deal, and was necessary in preparing 
lEPs by 1001 of the respondents in 1982. 



\ 



Table 7e 

The Extent School Personnel Felt the Summary 
Section in the Reports Was Useful in Preparing. lEPs 



1980 



1981 



1982 



Very l?seful. It Helped ■ 
A Great Deal and Was*^ 
Necessary . . ^ ' 


62 


r 

■ 74. ■ 


100 


i ■ 

3.59* 


39 


Somewliat Useful , But Not 
Really. Necessary 


28 ^ 


. 16 


' 0 


2,86* 


39' 


" Not Useful , and Was Not 
Necessary 


■b- 


' '3 


0 


1.05 


39 


No Response 


5 


'7 


' 0 


. 1.05. ^ 


39 



*£<.05 



The recommendations section and the diagnostic statement 

s 

1 ml- { 

section of the testing reports were evaluated as mor^^ posit fvl in the 

* . * \ ■ 

1982 data than in 1980 but the degree of change** or improvement was 
slightly less than significant. The change suggests a trend that 
these 'two sections were more effectively meeting the needs of school 
persoffn^l and were dbniidereo'more useful in preparing IE Ps in 1982 
than 0980 (seei Table 7f and 7g). 



Table 7f 



The Extent School Personnel Felt the Recommendations 
In the Reports Were Useful in Preparing lEPs 



Responsfe 



1980 
% 



1981 
% 



1982 
% 



1980-82 



OF 



Very Useful, Jt Helped 
A Great Deal artd. Was 
Necessary „ 


67 


79 


90 


1.88 


39 


Somewhat Useful, But Not 
Really Necessary 


18 


12 


0 


2.15* 


39 


Not Useful , and Was Not 
Necessary 


10 


■ 3 , 


5 


.61 


39 


No Response 


5 


6 


5 


0 


39 



►£<.05 



\ 



Table 7g 

The Extent School Personnel Felt the Diagnostic Statement 
In the Reports Was Useful in Preparing 'l^Ps 



Response 



1980 
% 



1981 



1982 
% 



Very Useful, It Helped 
A Great D^al and Was 
Necessary . ' 


71 


74 


• 

- ' 85 


1.10 


39 


.Somewhat Useful, But No^ 
Really Necessary 


19 


13 


10 


' ^3 


39: 


, Not Useful, and Was Not 
Necessary 


5' 


3 . 


0 


1 .^5 


39 


T^lo Response . ' 


5 


— —T 

10 


5; 


JO 


39 



1980-82 



DF 



£<.05' 



' • ' S.I 

c. Thb Extent School Personnel Felt the Psycho-Educational Testing 
Reports Were Accurate and Appropriate "V. 

'^lesearch literature conceraing psychological testing reports 

points out that the most fmport^nt information is contained in the 



findings and recommendations sections (Fischer, 1976). To improve 
the test report findings and recommendations, school personnel were 
asked to evaluate the accuracy and appropriateness of verbatim 
statements from these sections. 

Table 8, page 65, presents summary data evaluating the extfent 
school personnel felt that verbatim recommendations from the psycho- 
educational testing reports were at the appropriate level of 

specificity', provided new information, were realistic given available 
> 

resources and were useful in developing students' lEPs. 

the first evaluation question addressed the appropriateness of 
the specificity of each verbatim recommendation. Although the 
percentage of responses indicating verbatim recomnendations were 

• , • ^ 

"vague" or "too specific" decreased during the study, the percentage 
indicating the recofnmendations were "appropriately specific" 
increased only slightly. This change was not significant. 

' Table 8a 



The Extent School Personnel Felt the Recdmmendations 
In the Reportis Were Appropriately Specific 



Response 



1980 
% 



1981 
% 



1982 
% 



1980-82 



OF 



v ■ 

^ Appropriate ' 


\ — 

X' 84 


93 


88- 


.89 • 


236 


. , r 

Too Specific i 


5 


2 - 


6 


.34 


236 


Vague p.: 


n 


5 


6 


' 1.40 


236 


No Response 


■ 0 




0 


0 - 





£<;05 



One frequent criticism of psychological testing reports is 

' ~ • -J ■ " " ^ ■ • 

that n€|^ information* is- not provided users. School ^aersonnel often 



V 



64 



Table 8 



School Personnel Evaluation of the Accuracy and Appropriateness 
of the Recommendations in the Psycho-Educational Reports 



Percent of the Recommendations 
Rated in Each Cateqory 



Chuska 
1980 



N = 12 R " 75 



1981 N =^'-13 R = 70 

1982 - N = 7 R = 38 



Tohatcht 

1980 - N = '5 



R = 35 



1981 - N 16 R « 99 

1982 - N = 8 R = 41 



Gredsewood 

1980 - N = 4 R = 20 



Is the recommendation' 
at an appropriate 
level of specificity? 



.a 



81 



89 



92 



97 



93 



93 



70 



u 

■H 



10 



15 



20 



0 



0 



Based on your expe- 
rience with this stu- 
dent, does this recom- 
mendation provide you 
with new information? 



8 

■H 
4J 



% 



15 



21 



17 



13 



32 



30 



S 



1 



56 



87 



55 



23 



51 



66 



35 



g 



17 



20 



19 



25 



% 



12 



16 



40 



17 



10 



Given the resources, 
how real istic is 
this recommendation? 



in 



X 



64 



81 



68 



100 



77 



76 



23 



10 



26 



15 



2» 



u 



24 



10 



11 



0 



25 



How useful was this 
recommendation in 
developing this stu- 
dent's individualized 
education program? 



9 rH 

> o 

n 

% 



> 



65 



47 



63 



100 



58 



98 



45 



21 



41 



11 



18- 



35 



13 



24 



1981 - N = 9 R = 58 



98 



0 



14 



47 



19 



21 



67 



21 



12 



55 



34 



1982 - n » 5 R » 30 



73 



10 



17 



23 



63 



13 



50 



33 



17 



100 



Totals 

1980 - N = 21 R = 130 



84 



11 



0 



18 



44 



19 



16 



10 



72 



18 



1981 - N = 38 R » 227 



93 



12 



6\ 



.15 



' 12 



76 



15 



54 



30 



1982 - N » 20 R « 108 



88 



0 



62 



5 



67 



18 



87 



N = The number of school personnel Interviewed.. 

R = The total number of recommendations that were taken verbatim from the testing reports 

for each school each year. ' : " ^' 

% = The percentage of^he total number of recoinriiendat iorts "R" in each category each "year. 



complain that they can predict the psychologist's findings and 
recormiendations long before the child is referred (Fifield, 1982 ).> 
If testing reports are to be relevant and ijseful, they should prov^ide 
new information to the users or confirm their professional judgements 
about students' academic needs. Therefore, to improve the- usefulness 
and relevancy of test report inforfnation,' activities of the project • 
should result in increased new information for school personnel or 
information confirming prior suspicions or judgments about students' 
needs. ♦ r 

Table 8b indicates that during the course of the study, 
reviewers felt ther^ was a significant reduction in the* percentage of 
recommendations already known to the users of reconimendations that 

• * 

they were, certain would be j^conmended. Concomitantly there was a 
significant improvement in the percentage of recommendations and 
findings which confirmed prior judgements and suspicions of school 
personnel. Although there was a slight increase in the percentage- of 
schocfl personnel who felt that th^ recormiendations, provided 'jjgB 
information.^ this incj'fiase was not significant. ' , 

Table 8b 

The Extent School Personnel Felt the Recommendations In the 
Reports Provided New Informat^n About the Students 



198G 



1981 



1982 



New' Information . 




12 


26 


'1 *.48 


236^ 


Confirmation . 


44 : 


. 61 


62 


2*82*. 


236 


No New" Information 


19 


15 


7 


2.84* 


236 


No Response ' , 


15 


12 


-5 

1 '—^ 


' 3,47* 


236 



*p<.05 



'66/ 



Recommendations contained in psychological testing reports are 
. often criticized because school personnel feel they are unrealistic 
wi'th the existing resources on the reservation. Table 8c provides 
data in which school personnel ev.aluated how realistic verbatim 
recommendations from the reports were with the existing resources on 
the reservation. These data suggest that efforts to improve Tecormien 
dations to make them more realistic were not successful. A signif- 

icantly higher percentage of scliool personnel rated recommendations 

i 1 

as completely unrealistic at the^ conclu'is'lon of 'the project than at 

/■ ^ ■ ' ■ . ■ 'I 

the beginning, and the percentage of recommendations rated as 
realiistic reduced slightly. i > 



Table 8c 

The Extent School Personnel Fel>t- the Recommendations Were 
Realistic Considering the Given Resources 



1980 
% 



1981 



1982 



t 

1^980-82 



DF 



Realistic 


71 


76 


67 


.66 


236 


Somewhat Realtltic 


16 


15 


18 


.41 


236 


Completely Unrealistic 


3 ' 


2 


13 


«2.80* 


'236 


No Response 


10 


7 


2 


2.71* 


236 



*'£<.05 



; Table 8d provides data that evaluates the extent school 
personnel fejt the reconwendations we're useful 1n^ developing 
students' lEPs. The table shows a significant increase in the^ \ 
percentage of responses in the most favorable category at 
comple1?ion of the study. These data appear inconsistent with the\ 
data cOntained^n Table 8c, for a,, literal interpretation would 



suggest that even though school personnel felt the recommendations 
were less realistic with existi,ng conditions and resources, they were 
more usefuf in helping them develop the students' lEPs. 



L 



Table 8d 



The Extent. School Personnel Felt the Recownendatlons Were 
Useful In Developing, the Students* Educational Programs 



Response 


1980 
% 


1981 
% 


1982 
% 


t 

1980-82 


DF 


Very Useful 


' 72 


54 


87 


2.94* 


236 ^ 


Somewhat Useful 




30 


5 


3.28* 


236 


Not Very Useful 


8 . 


5 


8 


0 


236 


No Response 


2 


11 


0 


1.63 


236 



*£<.05 

<j. Evaluation by School Personnel of the Clarity, New Informat 



• ■ / 

ion, « 



Accuracy, and Usefulness of the Findings onjesting ReportT 

To determine the clarity, new Information, accuracy, and 
usefulness of the findings of testing reports, reviewers were asked 
to respond to four evaluative questions. These data are presented in 
Table 9, page 69. Although certain sections and criteria improved 
significantly, a consistent trend for Improvement was not indicated 
by school personnel evaluating the results section of "the testing 
reports, j ■ v 

• ^At%ie beginning of this study, slightly over half of the ^ 
repondehts reported that they^ould understand "very clearly" what 
the p»ychoVog1sts meant when they read the- verbatim findings. At the 
erid of the Jfcjly*'' approximately 90X indicated they could understand 
" V ery , c 1 e ar i y"wh at the psychologists meant by a verbatim statement 



68 



«9 



Table 9 



\ 



School Personnel Evaluation of the Accuracy and Appropriateness 
of the Findings In the Psycho-Educational Reports 

i 

A ' B 





Can you understand what 
the psychologist means 
by this statement? 


Does this statement 
provide you with new 
information about the 

student? 


Based on your expe- ' 
rie/ice with this 
student, Is this 

'statement correct? 


Was this statement 
useful in planning 
this student's 

educational program? 


: 

Percent of the Findinqs 
Rated in Each Category 


Very Clear 


Sonewhat 
Clear ' 


Hard to 
Understand 


Inpossible to 
Understand 


No Response 


New Irtf orma- 
tior Provided 


Suspicion 
Verified 


No New 
Information 


No Response 


Correct 


Partly 
Correct 


ij 

i 
§ 


No Response 


Very 
Useful 


■p 

C At 

M 




f 




% 


X 


% 




% 


X 


X 


ft 


X 


% 


% 


% 


X 


% 


X 


X 


X 


Chuskfl 

1980 - N = 12 F = 65 


83 


14 


3 


0 


0 


40 


46 


' 9 


5 


69 


25 


3 


3 


72 


15 


11 


2 


. "1981 - N » 13 F « 86 


80 


13 


5 


2 


0 


• 40 


50 


9 


1 


67 


15 


2 


15 


57 


27 


7 


9 


I9H2 - N = 7 F = 43 


91 


7 


2 


Ot 


0 


53 


42 


5 


0 


77 


.2i 


0 


0 


58 


r 

23 


2 


16 


Tohatchi 

1980 - N = 5 F = 30 


43 


50 


• 7 


0 


0 


100 


'0 


0 


0 


40 


J 

0 


0 


60 


• 100 


0 


0 


0 


1981 - N = 16 F - 109 


70 


26 


3 


0 


2 


50 


37 


14 


0 


64 


28 


0 


. 7 


71 


27 


3 


0 


1982 - N = 8 F = 4fl 




8 


0 


0 


0 


46 


48 


6 


0 


100 


0 


0 


0 


100 


0 


*— ; f 


— 

0 


Greasewood 

1980 - N = 4 F ' 22 . 


50 


36" 


14 


0 


:<< 

0 


32 




* 

14 


5 


68 


18 


9 


5 


36 


55 


5 


5 


/ r9Hl - N = 9 F = 57 


88 


11 


2 


0 


iO 


42 


40 


16 


2 


54 


9 


5 


32 


65 


" 19 


4 


12 


' JQfl? - N s n r s m 

• ^ Joc — n * 3 r * jn 


92 


5 


3 


0 


0 


32 


50 


18 


0 


71 


29 


0 


0 


100 


0 


0 


0 


Totals 

1980 - N - 21 F ■= 117 


67 


27 


6 


0 


0 


54 


35 


8 


3 


6? 


17 


3 


18 


73 


• 19 


^ 


1 


1981 - N = 38 F = 252 


78 


19 


3 


0 


0 


44 


42 


13 


1 


64 


19 


2 


15 


65 


25 


4 


6 


1982 - N * 20 F » 129 


91 


7 


2 


0 


0 


44. 


- 47 


9 




84 1 


16 


9 


0 


94 


6 


0 


0 



M = The number of s r .ho ti t personnel interviewed. • 

of findinqs tliat weroj. taken verbatim from the testing reports 

for each school each year. 

The percentage &f the total number of findings "F" in each category each year. 



.ER?C 



90 



91 



in the. f in(^ings section. This change in the rating^ was significant 

(see Table 9a). • • 

Table 9a * ^ 

The Extent School Personnel Felt They Understood 
What the Psychologists Were Saying in the Reports 



Response ' 


1980 
% 


1981. 


1982 
% . 


t ■ 
1980-82 


DF 


Very Clear 


67 


78 


91 


4.78* 


244 


Somewhat Clear 


27 


»19 


7 


4.27* 


244 


Hard to Understand 


6 


■ 3 ' 


2 


1.59, 


244 


Im|iossible to Understand 


0 


. 0 


• 0 


0 




No Response 


0 


0 


• 0 


0 





^p<.05 



Reviewers did not favorably rate the testing report as 
providing- new information for them; Table 9b indicates that the 
percentage of respondents rating specific findings as providing new 
information decreased slightly over the course of the study. This 
was accompanied by a slight increase in the percentage of reviewers 
who felt that the findings section provided more confirmation of 
previous suspicions. Neither of these trends, however, were 
significant (see Table 9b). 
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Table 9b 

The Extent School Personnel Felt Statements In the 
Reports Provided New Information About the Students 



1980 1981 ' 1982 g t 
Response X % % H 980-82 DF 



New Information 
Provided 


54 


44 


44 


* 

1.57\ 


244 


Suspicion Verified 


35 


42 


47 


1.93 


244 


No New Information I 


8 


13 


9 ' 


.28 


244 . 


No Response 


3 


1 


0 


1.90 


244 



£<.05 



Table 9c presents data concerning t<he accuracy of verbatim 
statements taken from test rep6H finding's. These data indicate that 
the .^percent age of reviewers rating the stcitements in the most favor- 
able category improved significantly. In addition, the percentage of 
^ . reviewers that reported findings were inaccurate dfecreased, along 
with a significant reduction in the number who did not respond. 

' Table 9c 

The Extent School Personnel Felt the 
Statements in the Reports Werie Accurate 



Response 


1980 
% 


1981 ■ 
% 


1982 
% 


t 

1980-82 


DF 


Correct 


62 


64 


84 


3.98* 


244 


Partial ly Correct 


17 


19 


16 


.21 


244 


Incorrect 


3 


2 


0 


1.90 


244 


No Response 


18 


15 


0 


5.07* 


244 



*£<.05- 
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Verbatim findings from the testing reports were rated as 
significantly more useful in developing lEPs at the end of the study 
than* at the beginning. Furthermore, in 1982, all reviewers rated the 
findings as being either "very useful" or "somewhat useful" (see • 
Table 9d). 

. '• ^ 

■ Tible 9d 

■ ■* 

The Extent School Personnel Felt Statements in the Reports 
Were Useful in Planning the Students' Educational Programs 



1980 1981 1982 t 
Response % % % 1980-82 DF 



Very iJ_seful 


73 


65 


94 


4.56* 


244 


0 Somewhat Useful 


,19 


25 


6 


3.11* 


244 


Not Useful 


7 


4 


0 - 


2.97* 


244 


No Response 


1 


6 


0 


1.09 


244 



*£<.05 



e. The Extent School Personnel Felt That the Overall Tes'ting Report 
Clearly Stated Testing Results and Was Appropriate 

/^^ 

Table 10, page 73, presents summary information indicating the"' 
clarity and appropriateness of the psychological testing reports. 
Respondents were given, and asked to read a typical testing report of 
a chil<l with whom they were acquainted. They were asked to rate six 
questions abqut the report from most "favorable to least favorable < 
(clarity of the report, usefulness in determining placement, 

i . ' * 

frequency-of unexplained technical words or phrases, compari son' of" 
the report with other reports, how well the report addressed the ' 

referral question, and how adequately the examiner took into , 

■ ^ '" 

consideration culture arid social factors). 



* ' . Table 10 

Evaluatl^on by School Personnel of the Clarity and Appropriateness 
' ■ of the Overall Psycho-Educa^tlonal Testing Reports 





¥ 


• 

Chuska 




Tohatchi 


'Greasewood 


' Total 


N » 12 
% 


1981 
N » 13 
X 


1W 

•N « 7 


1980 
N * 5 
X 


1981 
N «= 16 
X 


1982 
N » 8 
X 


1980 
N " 4 
X 


IdSI 
N » 9 
X 


mr 

N » 5- 
X 


N -.21 
X 


1981 

N » 38 
X 


1582 
N = 20, 
X 


How clearly did this.rfeport state 
thi-^ student's testing results? 

Very Clear, 1 Understood Everything 




92 


- 

86 


"C " 

60 


68 


88 


* 

.25 


. 

88 


• 

80 


67 


- 

82 


i 

85 


Moderately^HCIear, There Were Very 
Few Things I Couldn't Unders^tand 


16 


8 


14 


40 




12 


75 


11 


20 


'33- 


18 


15 


Moderately Unct^^ar, There Were 
Several Points .^f. Couldn't Understand 


0 


0 


0 


0 . 




0 


0 


0 


0 


0 




0 


Not At All Clear» There Were Many 
Points I Couldn't Understand 

'■m^ — ' I ■ ^ , ^ 


0 


f 

0 


0 


■ .-0. 


0 


0 


0 


0 


0 


0 


• 

0 


i 

0 


No Response ^ 


0 


0 


0 


0 


0 ^' 


0 


0 


0 


.0 


0 


*0 


0 


Was this report useful in determining 
this student's placement? 

Very UseVol 


58 


46 




4 

8^ 


43 


88 


?6 


44 


80 


57 


45 


75 


Somewhat Useful 


42 


46 


43 


*~ — 

2 


56 




75 


55 , 


20 


43 


53 


25 


Not Useful 


0 


8 




0 


0 


0 


0 


0 


0 


0 


0 


3 


0 


No Response 


0 


0 


0 


* 

' 0 


0 


0 


0 


0 


0 


0 


0 


0 


How often did you find technical words 
or phrases which were not ad.fequately 
explained? 

Little Jargon Was Used 


* 5fl 


62 


57 


80 


38 


• 

75 


100 


67 


6 


0 


71 


. 53 


« 

65 


Some Jargon Was Used, Rut the Report 
Was Usually Understandable 


33 


31 


43 


20 


\50 


12 


0 


33 


40 


24 


39 


30 


Substantial ^Jarqon Used, 
Hard to Understand * 


0 


0 


0 


f 


- 1 

6 


\ • 
12 


0 


0 


0 


0 


3 


5 


Frequ^ Use of Jargon, Extremely 
Difficult to Understand 


8 


0 


0 


0 


^ ^ ■- 

X 

0 


0 


* 0 


0 




5 


0 ■ 


0 ' 


No Response i 


0 


8 


0 


0 


6 


0 


0 


0 


0 


0 


5 


0 



^ Tftbltf 10 (tontlnued) ^ 





Chuska • 


Tohatchi, „ 


• Greasewood 


' . Total 


,N » 12 
% 


N - 13 
t 


1982 
N - 7 
t 


1980 
N - 5 
X 


1981 
N - 16 
X 


1982 
N - 8 

■ "X 


^980 
N --4 
.X 


i^er 

N - 9 
X 


198? 
N - 5 


1980 
N - 21 
X 


T981 
N - 38 
X 


1982 
*N - 20 
X 


D". How does this reoort compare with other 
reports you have seen in the past year? 

% 

^ About the Same ' 


16 




43 


0 


• 

' 50 '\ 


f 
» 

8^' 


- 

• 

.100^ 


• 

*66 


100 


,'29 




». 

75 


* • 

Worse Than the Others 


0 . 


0 


0 


■ 0 


0 


0 


• 

0 


•« 0 


0 


0 


0 


a 


, Better Than the Others * ^^^^^^^ ^ 


75 


39 


57 


100 


50 


12 


0 


11 


0 


67 


37 


25 


No Response 


8 


- 8 ■ 


0 


0 


0 


0 


■m 

0 


22 


0 


^ 5 

ft 


8 


0 


E. no these recommendations address the 
questions raised by the refenring 
, teacher? . 

. * 

Referral Question Well Addressed \ 


83 


70 


8fi 


• 

80 


• 88 


r 

100 


25 


0 


80 


71 


- 

^79 




* Referral Question Partly Addressed 


16 


8 


14 


20 


6 


0.' 


75 


11 


> 20 


V_i9_ 




. lo' 


* Referral Question Not Addressed 


0 


0 


- 0 


0 


. iL 

6 


0 


0 


11 


0 


0 


3 


0 


No Response * ^ 


• 0 


23 


0 


P 




f 

' 0 


0 


0 




• n 




F. Do you feel that the examiner gave 
appropriate considerations to societal 
and cultural factors<^1n this student.'s 
case? ^ 

Completely Appropriate Consideration 


■ ■ 
42 


\ 

53 


57 


60 


J 
50 


75 


• 

2 


33 


60 


- 

. 43. 


— 1 — 

47 


t 

65 


Partly Aopropriate Considerat-ion 


,42 


30 


' 43 ' 


40^ 


f " 


25 


75 


66 


40 


48 


. 45 


35 


^ P^tly Inappropriate Consideration 


16 


15 


0 


0 


6 


0 


0 


<• 

0 


0 


10 


c — 
8 


0 


Completely Inappropriate Consideration 


0 


0 


0 


• 0 


• 0 


0 


0 


0 


0 


0 


0 


0 


Ho Response 


0 


0 


0 


0 


V 0. 


0 


0 


0 


» 0 


0 


0 


0 



As^ table lOa indicates, althougy^responderits felt that the 
clarity of the testing reports Improved during the course of the 
study, t-his increase was not statistically significant. 



1 



Response 



Table 10a 

Jhe Extint School Personnel Felt the Reports 

Clearly' Stated the Students' Testing Results ^ 

' 1980 1 981 ^ iW t 

S % % % 1980-82 OF 



Very Cl^^^Tf = \ 
Understood EverythlngL 


' 67 ' 


82 


85 


1.38 


'39 


Moderately Clear, There 
Were Very Few Things I 
. Ciiuldn't Understand 


33 


18 


15 


1.38 


39 


Moderately *Uncl ear. There 
Were Several points I 
Couldn't Understand 


0 


0 


0 


0 




No 'Response 


< sO 


0 . 


0 


0 





£<.05 



Table 10b reports the rating of school ' personnel on t^e over- 

all usefulness of the reports in determining the students' placement. 

\ 

As in the previous question, there was an increase in the percentage 
of reviewers who rated the overall usefulness of reports in -the most 
favorable categpry, but the change that occurred between 19^^ and 
198^ was not significant. *> ' 




99 
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Table 10b * 

The Extent School Personnel Felt the Reports 
Were Useful In Detemlnlng Students* Placement 



1980 



1^81 



1982 



- y 

Very. Useful 


57 


45' 


/ 75 


1.24 


39 


A \ 
'Somewhat Useful * 


43 ^ 


52^ 


25 


1.48 


39 


Not Useful 


i 


3 


0 


0 


m w 


No Response 


0 


0 


0 


0 





)t.05'- 



School personnel often complain about jargon and technical 
words used in testing reports that are not adequately explained 
(Rucker, 1967b and Shively & Smith. 1969). Table 10c presets data 
determining the extent school, personnel feK reports contained 
such jargon. It indicates that little- jargon w6s i^d, and what was 
used, was easy to understand. Ovei^ the course of the study, it 
appears that reviewers felt ther? was a slight increase in the amount 
of jargon used, but that such jargon and technical words were easier 
to understand, due to explanations in the report. This change, 
^ however, was not. statistical lyj significant. 



V) 



Table IQc 



The Extent School Personnel Found Technical Words 
or Phrases In the Reports Difficult to Understand 



Response 



1980 
X 



Little Jargon Was Used 


\71 


53 


65 




^ 39 , 


Some Jargon, But 
Generally Understandable 




24 




30 : 


.43 


39 


Substantial Jargon Used, 
Hard to Understand 


0 


3 


5 




39 


— - - 

Frequent Use of Jargon, 
Extremely. Difficult to 
Understand 


'5 


* 0 


0 


. 1.05 


39 


i 

No Response 


0 ^ 


5 


. 0 


0 





1981 
% 



1982 
% 



1980-82 



DF 



£<.05 



One approach for evaluating testing reports is to compare them 
with othter reports that 'school personnel . have used in. the past. 
Table lOd presents the resljlts of this comparison. ^ 

When interpreting the data of Table lOc, it should be noted that 

the 1980 reviewers compared' test:ing reports that were part of this 

<■ ■ • 

study with reports prepared "by individuals not associate'd with the 

Excepijiona^ Child Center; thus, this second group of reports con- 

taihed a different testing format and different procedures used for 

test administration. In 1981 and 1982, the reports being evaluated 

♦ 

were compared with the previous year's reports which were a part of 
this study. * 

Table lOd -shows that in 1980, 67% of the reviewers felt that the 
testing reports prepared by ECC psychologists were better than the 
reports prepared by individuals not associated with t'he EC^' 28% felt 



V 



77 



101 



that they were about the same, and no one f git that they were not as 
.good as other reports they had used. 

The 1981 testing reports were compared with those from 1980 and 
followed the same general fdrmat of the 1980 comparison. The per- 
centage of reviewers who felt that the 1981 reports were better than, 
.the reports of tlie previous year' dropped to 37%, ind 57< felt that / 
they were about the same. The 1982 data suggest that only 25%^ the 
reviewers felt that the 1982 testing reports were better than those 
provided in 1981, and 75% felt that they were about .the same. , 
AltlTough the change reported between 1980 and 19^is .significant, it 
is the change that occurred between 1981 and 1982 that is of pjirtic- 
.ular ifhportance to this study, -fhis change, although positive, was 
not significant. 



Table lOd 

The Extent School Personnel Felt the Reports 
Compared With Other Reports Seen in the Previous Year 



. Response 


1980 
% 


1981 
% 


1982 
% 


t 

1980-82 


' OF 


Better Than Others 


4 

67 


37 


25 


2.98* 


39 


About the Same 


^8 


55 


75 


3.41*. 


39 


Worse Than Otiief*-^ ' 


0. 


0 


0 


0 




No Response ^ — - 


5 


8 


0 


1.05 


39 



*'£<.05 



Another complaint of school, personnel of psychological testing 
reports is that they do not address the question asked by the -person 
referring the child (Rucker, 1967a). '^^^'^^ 'presents data which 



addressed the quest;Uio of how adequately reviewers felt that the 
testing reports*; responded to the referral questions. • • . 

In 1980, 711 of the reviewers rated the reports as addressing 
the referral question very well (i/i the most favorable category), and 
in 1982, 90% rated tTie reports in the most favorable category. ^ 
Although this finding was positive and encouraging, the change 
reported was not statistically significant. 

• .a 

Table lOe 

The Extent School' Personnel Felt the RecomnendatlOns 
In the Reports Addressed the Referral Questltfn , 



1980 



1981 



1982 



Referral Question/ .Wei 1 
Addressed ■ ^ 


71 


i 

78- 


90 


t.59. 


1 

.39^^ 


Referral Question 
Partly Addressed 


29 


• :^ 8 


10 


1;59 


39 


Referral Question 
Not Addressed 


0 


3 


0 


0 




No Response, 


0 


11 


0 


0 





p<.05 



School personnel, particularly those working with minority 
studjents, are often critical of psychol og'i sts and skeptical of 
testing reports that do not give adequate consideration to social and 
cultural factors unique to the minority population with which they 
are wSS^king (Oakland, 1977). This type of criticisft often results in 

* 

school personnel dismissing testtng results as being bias. 

T 

, Table lOf provides drfiii concerning the reviewers opinions df the 
extent the psychologists considered the social and cul^ral factors 



unique ^to the children being evaluated. In 1980, 43% of the 
reviewers indicated that the^ felt the examiner had. completely and 
very' appropriatel-y considered cultural ^^Kd social factors. In 1982^ 
65< of the reviewers rated the reports in the most favorable 
category. This change suggestss that the intervention during the 
study caused psychologists to give greater consideration to social 
and cultural factors unique to the children being evaluated; however, 
the percentage of change was not statistically significant. 



Table lOf 

The Extent School P'iBrsonnel Felt the Psychologists Gave 
Apprppriate Consideration to Social and Cultural Factors 



Response 


1980 


1981 


.1982 
% ■ 


t 

1980-82 


OF 


Completely Appropriate . 
Consideration 


43 


'47 




1.45 - 


39 


Partly Appropriate 
Consideration 


47 


45 


35 


.79 


39 


Partly Inappropriate 
Consideration 


0 


8 


0 


0' 




Comp^let^ly Inappropriate ' 
Consideration 


10 


0 


0 


1.53 


39 


^0 Response ' 


*0 


0 


0 


0 





.05 • 



3. There Win Be a Greater Relationship Between the Instructional Program 
'Proposed in the lEP and the Recommendations in the PsycO'Educatlonal 
Testing Reports ■ ~ 

One major assumption of this study was that if the information 

contained 1n the psycho-educational testing reports ii relevant, useful, 

accurate, and realistic, such information will indeed be used by school 

personnel. However, even though an educator, parent, or any test result 



user evaluates a testing report as helpful, accurate, and relevant, such 
a rating does not provide evidence that they are indeed using these 
results (Ownby and Wall brown*, 1983). Goldman points out- that testing 
results may be used at different levels (Goldman, 1981). Fifield (1982) 
believes that if individual testing- results are utilized only for general 
orientation purposes, confirmatio/ of professional judgements,' or to 
satisfy curiosity, the investment in time and other resources is not' 

warranted.^ , • ^ ^ , " 

-i/ 

- . * 

The need„for individual testing is often justified on the basis that 

it is necessary and valuable in making placement and prbgrarmiing deci- 
sions in preparing the students' individual educational programs (lEPs) 
(Kabler, 1977). Some studies have proven that testing reports are used 



ext^sively in developing I€Ps for students placed 1n special education 
(Fifield, 1982; Kabler, 1977). In certain instances, the results and ; 
rec'ommendations sections of th^ testing reports provide a major source of 
data for the students' lEl's. .-fn-other instances^ it has been observed 
that the lEP committee ignores individual test findings and recommenda- 
tions. The lEP, regardless, provides a document of placement and program 
recommendations v^ich can be compared with the findings^ and recommenda- 
tions on individual psycho-edapational test*! rjg reports. For this study, 
the degree of concurrence between findings and nicommendations on lEPs 
and testing reports served 'as an index of the degree school personnel . 
used testing reports and the purpose for which they were used. 

Data to compare the findings, and recommendations contained on LEPs 
and testing "reports were obtained by examining the findings and recommen- 
dations contained on a sample of student lEPs and testing reports. The 
degree of congruence was then determined between the two sets of data. 

. . .si. . 105 



These data .were coHectecj for each observation period of the study. The 
student population from which" to draw the samples consisted of all the 
students placed In special education In the target scliools. 

Reason for -Referral . A prel 1m1 nary step before comparing "the 
relationship between lEPs anftSstlng reports was an analysis of the 
findings and recommendations contained In the psychofoglcal testing 
reports for the students" placed In special education of the target ' . _ 
schools. The data presented' on Table Vl, page 83, Indicated changes 'fn 
the reasons c^jl 1 dren 'were referred for te\tin9 during the' course of the 
study. Fpr the 1980 testing,- behavioral and health problems accounted 
for a very small percentage of the referrals, but for the 1981 testing, 
behavioral and health problems accounted for 24% of the referrals. In ^ 
1982, behavlcy-al and/or health problems did not singly account for aijy . 
referrals; however, behavioral problems combined with academic problems, 
accounted fbr 27% of the referrals. ' 

Clarity ofJReferral Question . The need for a clear and precise . 
referral question was a problem fqr this study. .If the person^ making the 
referral does not know what information Is needed or wanted. It is * 
difficult for them to appropriately critique a testing report and 
determine if the report provides information that Is helpful. For this 
reason, the referral- quest ipn was stressed in the inservice training. 
For' 1980, Table 11 ind'icates that 56% of the referral questions were 
judged as good,^nd 4U-as-£alru— XheH Wc e ntag e of goo d raf a rral - — 



f 

questions increased in 1981 to 67%, and those referral questions fudged 
as fair decreased to 26%, The 1982 data, however, indicate that referral 
questions were fudged less favorably at the conclusion of the study than 
they were at the beginning; ' • 

■ V / I * ■ 



Table 11 

Referral and PUceMnt Infonutlon FroM the Testing 
V Reports of Students PI teed In SptcUl Education 



- Item _ ' » 


Totals 


1980 
■ 98 

' t ' 


1981 
N - 42 
< 


1982 
N - 33 


1 PTA^nN POP fiPFFRRAI 

A Af* jkfiom'i i* PirnHliMTi 

i\m nCducvnii* rruuictii 4 


', 93. 


79 


70 


B. BMiavloral Problem 


g— 


17 


0 


C. Health Problem ' . 


- — r~ 


7 


0 


D. Academic and behavioral Problem > 







27 




0 


— "0 — 


3 


9 riAOITY OP TMF RPFPRRAI fVIP^TION 

i ' ^ - 
A. Poor, Vague and Hind ear Quest 1 oh 


2 


7 


12 


, B. Fair, Adequately stated Question 


41 


26 




C. Good. Men Stated Question 


gg — 


67 






c 


0" 

V 






98 


76 


85 


B. Mental Retardation 


1 


14 


-4- 


C. Emotionally Disturbed 


0 ' 


3 




D. Other 


0 


5 


'3. 


4. PLACEMENT CLASSIFICATION 
A. Self-Contained 


1 


10. 




B. Resource ftoom 


78 


■■■ .45.' 


73" ■■ 


C. Small Group 


0 


33' 


18 


DL. One-to-One 


0 


12 


3 


v. Other 


*■ " 


— ■*" ' 


- — T~ 



■i 



( 



*This data not reported for 1980 and 1981 
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Interpretation of this data suggests that the inservi,ce training had 
little effect on how well referral questions were stated. Poor, vague, 
an<J unclear referral questions slightly increased over the -course of* the 
.study. However, thie director of special education and other school 
' personnel attributed this increase to the difficulty of getting referral 

forms, and tbe logistics process of preparing and submitting referrals 

» * ■ ■ ' 

rather than the* effectiveness of the inservice training. 

Recommendation, for Placement . Of the children in the three target 
schools placed in special education, approximately 36% were classified as 

' . . ♦ 

■learning" disabled. This high percentage of children classified as 
learning disabled can be explained. As noted in the literature, most ; 
psychologists »and lEP committee members are sensitive to the issues of 
' bias in testing, inappropriate placement, labeling, and other practices 
which may discriminate. The special education classlfication of learning 
disability Is felt to contain the least amount of stigma. When 
psych'b'logists and lEP committee members had misgivings concerning a 
classification becaiuse there was not clear or consistent evidence of 



4^ which handicapping category was most appropriate, yet there was apparent 

evidence that the child needed individualized help, this dilemma was 
^ , resolved by selecting learning disabilities as the classlficatioh of 
choice. In a similar matter, to comply with regulations that require' 
children to be placed in the least restrictive environment, very few 
students were recommended for self-contained placement. The majority of 

students were recommended for placement in a resource room, and a^ 

.In- 
significant number of children were recommended for small group type 

instruction. 
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The relitionship between the recommendations contained in the 
students' lEPs and their testing reports was determined' by using Pearson 
Product 'Mortent Coefficient of Correlation* In addition, -the specifi*pity 
of recommendations, i.e., no recommendation, general /broad, or detailed 
and specific rlkjmmendations, was factored in the correlation coefficient 
and a percentage comparison was made between the recommendations 
contained on the testing reports aod the lEPs. 

Relationship Between Short-Term Recommendations on Testing Reports 

I 

and lEPs . Table 12, page 86, presents the relationship between 
short-term recommendations contained in the testing reports and those 
contained on the lEPs. Ten categories of short-term recommendations were 
identified and- recorded on the lEPs and the testing reports. The 
percentage of each recommendation for each category, along with its 
specificity rating is presented on Table 12. In addition, the 
correlation coefficient between the lEPs 'and the testing reports is 
presented on Table 12. 

The correlation coefficient between short-term goal recommendations 
on lEPs and testing reports indicates that the relationship improved i 
betwe"en 1980 and 1982. The average change was* .18, However, not all 
categories of short-term reconwendations improved. The categories 
showing the most significant increase include: math concepts +.50, 
language and auditory skills +.22, functional math +.28, reading 
comprehension +.24, social skills +.38, and visual motor skills +.28. 
The learning skills alsO showed an increase, but not at a significant 
l^el. The relationship between IE P and testing report recommendations 
in reading mechanics -.03, math combinations -.13, and other recommenda-* 
tions -.10 was less in 1982 than in 1980. None of the reductions In 

109 



Table 12 



Relationship Between Short **tenii fteconjiendat ions Contained on tKe Testing Report and the lEP 

.- ■ ■ ■ ^? yv- v.. .1 ' i y . • . 



4 

Item 




No Recomnlei^clat^xin 
RrbyiilM 


} ;£eneraV« flroad 
RecqnvnieiMlatiQ'^^^^ Provided 


Detailed or Specific 
Reconinend«t1on Prpvldec 


r Between Test 
ReDort-'arfd lEP 


Change In r 


1980 
N « 98 
X 


1981 
N » 4 

.1 • 


I 


1982 
N • 33 
X 


* 1980 
N « 98 
X . 


I98I; 
N * 42 
X . 


1982 
N » 33 

■ X 


1980 . 
N « 98 
. X 


1981 
N ■ 42 
X 


1962 
N ■ 33 
X 


1980 
N » 98 


1981 
N « 42 
% X 


1962 
N « 33 
X 


Between 1980 
and 1982 


1. READING MECHANICS ^' 
Test Report 


1 1\ 

■ 10 


14 




49 


25 


12 


3 


65 


74 


49 


.45 


.37 


.42 


- — r ■ - - 

-.3 . 

• 


lEP ' 


17 






39 


47 


7 # 


9 






52 


2. READING COMPREHS((SION 
Test Report ^ 


43 


45 




15 


18 


12 


21 


39 


43 


64 


.39 

* 


.65 


.63 


+.25 


lEP A ^ 

1 f . I - - 1 .... 




48 ■ 


33 


30 


l9 


-w- 

0 


11 


33 


67 


3. 'MATH COHaiNATWNS ' 
Test fMnort J 




■%■*■!' 

19 ; 


« 

15 


23 


7 


0 


55 


74 \ 


V f 

79 


.35. 


.22- 


.22 


-.13 


IIP \ y 


— — m 
^ 31 . = 




85 


41 


2 


9 


26 


64 


V *6 


4. functional) 

T«St Rep/ 


WffH 


84 


98*1 


70 


8 


' 2 


0 


6 


0 


30 


r35 


.06 


.63 


+.28 


!fP f 




" IV art" — 
^ 89 


lOOf 


— "TO" 
79 




0 


' — fy"' 

0 


2 


0 


21 


5., MATH COXeS 
Test ^( 


rs ■ 
»rt 


79 


95 


33. 




5 • 




15 


0 


64 


* 

.18 


.00 


.68 ' 


+.50 




69 


IM 


f 46 


17 


0 


9- 


. 12 


0 


46 


6. langi/age and' auditory skills 

Test Report 


45 


( 33 


. 30 


21 


21 


15 


' . 32 


45 


55 


.04 


.00, 


.26 


*.2"2 


ifr — T 


" nil — 


\ 45 


?4 




10* 


9 ' 


Mr 

3Z 


45 


67 


•>-5-^st Report 


75 


71 


> 58 


12 


12 


9 


11 


17 


33 


.21 


.31 


.59 


+.38 


lEP 




SB 


55 


70 


6 


5 




4 . 


A 


?4 


8. LEARNING SKILLS-ATTENDING, ETC. 
Test Report 


81 


74. 


i 

82 


9 ' 


12 


3 


. 8 


14 


15 


.01 


.30 


.17 


+.16 


lEP 






93 




4 


5 


0 




2 


12 


9. VISUAL-MOTOR SKILLS 

Test Report . 


86 


64 


84 


,5 


21 


. . .9 ■ ■■ 


7 


14 


' 6 


..16 


.24 


.44 


+.28 


. TEF 






95 


91 




2 




6 


2 


3 


^. OTHER RECOMMENDATIONS 
Test Report 


91 


86- 


70 


5 


5 


0 


« 

4 


10 


30 


^13 


■ » ■ 

•41 


.03 


^ 

-.10 


MP 


74 


87 


S2 


15 


7 


0 


9 


7 


18 



00 



IIM 



O coefficient of .19 Is significant at the .05 level. The averaije chanqe for short -tern reconmendatloni was .18. ' 



\ relationship were significant,^ however, the sections showing Improved 

relationship werer si gnlflcant^^^ 

The dJi^a prjesented on Table 12 suggest that the Intervention through 

inservice training anO/ or changes in the psychological testing report. 

format and process did improve the relationship between the recommenda- 
r tlons for short-term go^ils contalnred on the testing reports and those 

■ > . ■ 

icontalned in lEPs. * 

Relationship Between Recommendations for Long-Term Instructional 
Goals Contained on Student Testing Reports and Their lEPs . Table 1*3, 
page 88, presents the relationship between recommendations for long-term 
instructional goals contained in the testing reports and those on the 
lEPs. As indjicated in 1980, the lEPs dijj^ not reporj long-term goals; 

thus, no relationship could be recorded. Furthermore, the percent of 

' ♦ • - ■ f'' 
psychological testing reports that contained long-term goals in 1981 and 

< " >f 

1982 increased and recoimiend at Ions became more specific- 

The opposite occurred with student lEPs; in 1980, there were no lEPs 



that contained .long-term tgoals, yet in 1981, 88< of the lEPs contained 
long-term goals for reading Instruction. . This was reduced, however, to 
19X In 19821 In 198U 71X of the lERs contained recommendations for math 
instruction, but 1n 1982, only 21X contained such recommendations. In 
the other four categories of long-term goals, fewer lEPs reported such 
goals and those goals reported tended to be very g/eneral . 

f 

During the course of the study, Inservice training activities 

focused on imp^-oving short-term goals In lEPs. The Impact of this 

emphasis 1s supported by the data presented in Table 13. The 

relationship between the long-term Instructional goals on the IE P and 

those in the testing report could only be reported for 1981 and 1982 
jlata. The average change was +.3 and was negligible. 



Table 13 



Relationship Between Recommendations for Long-Ters Instruction 
Goats Contaliied on the Testing Report and the lEP 



• ,. 

Item 


Ho Recomnenti 
Provided 


at Ion 


Sceneral, 8road 
Recommend at lot) Provldec 


Detailed or Specific 
Recommendation Provldec 


I t between Test 
X__a«l»<n't and lEP 




/ Change In r 
Between 1981 
'and 1982 


N - 98 
X 


1981 
N « 42 
X 


19«2 
N » 33 
X 


1980 
N - 98 
X 


1981 
N* 42 
X 


1982 
\N » 33 
X 


1980 
N « 98 
X 


1981 
N - 42 
X 


1982 
N - 33 
X 


N - 98 
X 


1981 
H - 42 
X 


T982 / 

.•^ 


1. LONG-TERM READING INSTRUCTION 
Test Report 


10 


5 


9 


73 


t 

86 


56 


15 


10 


33 




.14 


.21 


+.7 


TTBr — — ^ 


0 


12 




0 . 


83 


18 


0 


5 


3 


2^ LONG-TERM MATH INSTRUCTION 
/ Test Report 


1 

19 • 


t 24 


3 


72 


69 


46 


7 




52 


.00* 


.20 




• 

-,13 


' lEP 


0 


■V^9 




0 


S4 


12 


0 






3. LONG-TERH GOALS IN LANGUAGE 
• . OEVfLOPMENT 

Test, Report - ) 


45 




— ^ — 

39 


42 


74 


24 


11 


5 


36 




.13 


.31 


+.18 




0 




85 


0 


41 


9 


0 


5 


6 


4. LONG-TERM ^ALS IN SOCIAL SKILLS 

leSL KKpOiL 




43 




32 


57 


18 


12 


0 


27.3 


.00* 


,30 


.18 


• * * 






■ Bl ■ ■ 


91 


0 


19 


.6 


0 


0 


3 


5. LONG-TERM \\LS IN VISUAL-MOTOR 
DEVELOPMENT 
Test Report 


89 


83, 


t 

91 


7 


17 


9 


2 


0 


0 


7 

.00* 


.07 


.00 


-.7 


lEP 


0 


98 


ion 




2 


0 


0 


0 


0 


-< 




6. LONG-TERM GOALS IN SCIENCE, 
SOCIAL STUDIES, AND OTHER 
CONTENT AREAS 
Test Report 


77 


83 


61 


15 


12 


9 


6 


5 , 


30 


.00* 


•.11 




.26 


+.15 


m ^ — 


" 0 


93 


94 


0 


^ 


.3 


— r- 




3 



00 
00 



"^TFTTs tfata was not reported In the lEPs for 1980. 
The average change In r for long-term Instructional goals^liigs -^.3. 
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Relationship Between Recommendations for Specific Materials and 
Strategies, Sup^entental Services, and Criteria for Evaluating 
Effectiveness on the<i Testing Repots and the lEPs . The last three ar^as , 
on the testing reports 'and lEPs that requested specific recommendations 
included: (a) specific materials and instruction, (b) supplemental 
services, and (c) the criteria for evaluating the effectiveness df the 
.. program. These data are presented in Table 14, page 90. The 1980 ips 
did' not report recomnendatlons for ]/ese three categories, and in 1981 
less than 20X of the lEPs reported recommendations for these categories. 
This improved in 1982 with 58< of the lEPs reporting r|coninendat1ons for 
specific Instructional materials, 39% reporting recoiRdations for 
supplemental services, and 12% providing evaluation cj^fferiTTor the 
program's effectiveness. 

During the course of the study, the number of testing reports that 
contained reconinendations for these three areas increased, as well as the 



specificity of' such recommendations; however, the relationship between 
these reconmendations and those on the lEP decreased. As with long-term 
goals, the focus of the lEP inservice training offerred by the agency 
focused on short-term goals and comparatively little attention was given 
to recommendations in the above areas; thus rd^ec ting the focus, the 
correlations shifted from positive to negative between 1981 and 1982 with 
an averige change of -;49. This- relationship change was significant but . 
riot meaningful, in view of the facts as described above. 

The data collectors recording and evaluating the specificity of 
recommendations in §tudent lEPs and testing reports provi^e^ a rating of 
agreement between the testing r«wtrts and lEPs. As presented on Table 
15, the di^ee of Improvement in the relationship of findings on testing 



Table 14 

Relationship Between RecoimiendatloAS for Specific Naterlil or' 
Instructional Strategies* Supplemental Services* and Criteria for 
Evaluating Effectiveness Contained on the Testing Report and the lEP 



Item 


No Recommend 
• • Provided 


at ion 


General, Broad 
Recommendation Provfded 


Detailed or Specific 
Recommendation Provfdec 


r between Test 
Report and lEP 


Change In r 
Between 1981 
and 1982 


1980 
N « 98 


1981 
N - 42 
t 


N - 33 


1980 
N « 98 
X 


1981 
N - 42 
« 


1982 
N » 33 


1980 
N « 98 
% 


1981 
N » 42 

i 


1982 
N • 33 


1980 
N « 98 
X 


1981 
N - 42 
X 


1962 
N - 33 
X 


\. J(ECOHMENDATION OF SPECIFIC 
M)\TERrALS AND INSTRUCTIONAL 
STRATEGIES 
Test Report 


25 


14 


1 

18 


10 


10 


9 


■ 

63 


76 


^ 73 


.00* 


.10 


; ■ 

-.19 


-.29 


lEP 


0 


86 


42 


a 


i 12 


39 ^ 


0 


■ 2 


ifl 


2. RECOMMENDATION FOR SUPPLEMENTAL 
SERVICES 

Test Report 


43 


26 


15 


2 


74. 


12 


53 


0 


73 


.00* 

< 


.11 


-.32 




-.43 


TEP 


' 0 


91 


61 


'' 0 


5 




■ 5 


0 


30 . 


V 

3. CRITERIA FOR EVALUATING , 
EFFECTIVENESS ^ ' 
Test Report 


22 


5 


0 


8 


/ 

95 


. 24 


68 


0 


76 


.00* 


.70 


-.04 


* 


-.74 


lEP 


0 


98 




. 0 ' 


2 


15 


0 


L_JU 


3 



o 



* This data was not reported in the lEPs for 1980. 

The average change for material and strategies, supplemental services, and criteria was *.49. 
t 
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Tatfle 15 

V Overall Rating of How Hell the Test Insk Reports Agreed 
With the lEPs and the Quality of the Reports 



■ --k;^ 

Item 




Poor 


Fair 


Good 


^' -^^Ir^ct Agreement 


H • 98 
% 


1981 
W - 42 
% 


1982 
N • 33 
X 


1980 
N - 98 

i 


i9Br 

N « 42 
X 


W8? 
N - 33 
X 


1980 
•N - 98 
X . 


1981 
N « 42 
X 


1982 
N - 33 
X 


'1980 
N - M 
X 


1981 
N - 42 
X 


1982 
N - 33 
X 


1. Overall Rating of How Well 
the Testing Reports Agreed 
With the lEPs 


30 


26 


12 


44 ' 


41 


49 


23 


31 


24 




* 


15 


2. Ratinq of Overall ^iuallty 
of Testing Report 


, 4 


4 


0 


,40 


41 


12 


53 


52 


68 






/ 



This data not reported for 1980 and 1981. 



reports and lEPs was small. The evaluators felt that approximately 2^% 
' of the reports had good agreement with the lEPs and nearly half herf^air 
^greemeifitJ 

- " ' ' ' 

In analyzing the 1982 data. Instances v*ere Identified In which It 

■ ■ ■ 

was apparent ^th at the testings-report was '^almost excluslvelyj, to^ 
^ develop the lEP. .A new category (direct agreem^t) was added to the data* • 
and the frequency of direct agre^nent betweer«est1ng reports and IBP&>-- , 
Vfas recorded for 1982. This direct agreement occurred in ap"proximately 
15« of the reports reviewed in 1982. ^ . » ; 

* ■ ■ . 

Data col lectors %ere alsa asked to. provide a rating of the overall 
quality of the testing reports. These data are presented l^tfblti 15 and 
indicarte that data collectors felt there was a slgnlflcMt 'improvejnent Irf'Y ^ 
the quality of the testing reports over the .course of the^ 'sjudy. \ 



4. At the Completion of the Project, Referral Forms. Tartlnq^ftepof.tl, jlnd^ ?; 
-Ottier-Procedures Used to Communicate Individual Test Fincllngs '*and"^> 



gs 

Recommendations Win Have Been Improvecf " ^ \;° , ^--ry 

The data collected to address the above objective incTiuJ^d verla-V 

and written comments from Interviews with targettTlchooT personnel Viong 

with the Test Report Critique Form which contained both forc^/choice/«gd 



0 



open-ende^oomients* The Test Report Critique Forin asked reviewers to . 
evaluate the referral and report forms and other activities used in th^, 
agency to communicate test findings to users (see Appendix C). These 

• ' * . 

examples and cornneots were used to stimulate further Inquiries during the 
interview. . ■ 

The nature of the interview fadused on criticisms and suggestions 
"from school personnel for irtiproving t^Bsycho- educational assessment 
process and the test Irft) reports. The Wl y solicited opportunity for 
respondents to provide positive comments was to the question, "How did 



the testing report compare with other testing reports?" Individual 

coninents were t^ulated and the percentage of criticisms, reccimienda- 

tions, neutral rfiiponses and positive responses were calculated. These 

data are found on Table 16, ijage 94. 

The 1980 data indicate that 62% of the comments were critical of 

psychological testing. The percentage o/ critical coninents of psycholog- 

»*■ 

ical testing reduced during the study, but not significantly. In 1980, 
^criticisms focused prfmari'ly on the availability of reports, turnaround 
time (from referral to report), and the nature of the referral. In 1981, 
criticisms focused primarily on the validity of the findings /and cultural 
considerations. In 1982, most criticisms ad(Sessed the avai 1 ability ^of 
reports, cultural considerations, and the reality of recommendations. 

The recommendations in I960 focused on the format of reports, 
specificity of the recommendations, the assessment process, and issues 
concerning communicating testing results with staff. In 1981, most 
reconmendations addressed the issue of increased specificity of the 
findings and recommendations^ in the report ,ahd techniques for improving 
culture consideration, in the assessment of children. In 1982, the 
recommendations focused on the v'alidity of the findings, turnaround time, 
the assessment process and a vefriety of othec comnentS. 

Seventy-two percent of the complimei^ry comments in 1980 concerned 
the quality of the reports. Other areas .complfjnented inc^jded the 
sp^lficlty of recommendations, and the consideration, psychologists gave 
to cultural factors. In 1981, only 19< of the coninerits about the 
quanty o/^-the reports were complimentary. Culture consideration and 
validity M findings received W and 1456 of the compliments respec- 
tively* Alnl 1982, Z2% of the complimentary comments concerned the use of 



Table T6 

Sunury of Written and Verbal Coflwents of School 
Personnel Concerning Psychological Assessment 



4^ 



I 





1980 


1981 




i 


1982 




Totals 


Category of Written arid/or 
Verbal Cocnments 


1 

a 

X 


o 

i 

i 

X' 


\ 

■ o 
N-18 
X 


Is 

N-0 

^ X 


1 

•H 
U 
O 

X 


\ 

■ ' 

1 


1 
1 

0 


1 

■ 11 

■N«37^ 
X 


1. 

ill 
N-;?3 

X 


\ 

0 

.N-33 


\ 

N-1 
X 


• 

7 


I 

o 

N-9 
X 


It 
It 

t 


1 

0 
N-224 
X 


1 
1 
1 

N-106 
X 


1 

N-64 
X 


9| 

nth 

N-28 
X 


Referral 


13 


5 


0 


. * 


4 


10 


3 


4 


0 


0 


0 


20 


7 


7 


I 


7 


Coirtmunlcatlons— StafMniji : 


2 


r 


5 


* 


4 


0 


,0 


0 


Q 


0 


0 


0 


3 


5 


2 


0 


turnaround (Referral to Report) 


18 


0 


0 


* 


7 


6 


0 


0 


3 


12 


0 


0 


11 


5 


0 


0 


Availability of Reports 


:20 


3 


0 


* 


* 

10 


2 


3 


0 




0 


0 


0 


15 


2 


2 


0 


Format of Reports 


10 


21 


0 




1 


2 


S 


' 0 


b 


0 


11 


0 


4 


8 


5 


0 


Specificity 


a 


18 


11 


* 


9 


24 




' 0 


0 


12 


s 


0 


0 


9 


19 


3 


0 


Test Appropriateness 


3 


3 


0 


* 


1 


0 


0 


0 


0. 


0 


0 


0 


2 


1 


0 


0 


Culture Consideration 


5 


8 


11 


* 


14 


28 


19 


0 


15 


6 


11 


0 


11 


17 


16 


0 


Validity of Findinqs 


8 


8 


0 


* 


20 


6 


14- 


0 


9 


18 


22 


0 


14 


8 


11 


0 


Reality of Reconnendations 


• 4 


3 


0 


* 


6 


6 


3 


0 


12 


6 


11 


0 


6 


5 


3 ^■ 


0 


Technical Data and Jargon ( 


3 


3 


0 


* . 


7 


2 


14 


0 


» 


6 


11 


u 


6 


3 


9 




Jlnessment Process 


. 3 


IS 


0 


*. 


•■ 6. ' 


6 


11 


0 


6 


* 

12 


0 


20 


S 


10 -* 


*' 6 


4 


■bality of Reports . 


0 


0 


'72 


*- 


1 


0 


19. 


0 


6 


0 


11 


0 


1 


0 


33 


0 


Need More Training About Testin<f 




■ * 


* .' 


* 


0 


4 


0 


4 


0 


6 


0 


0 


0 


3 


0 


4 


Hore Fulliarlty Needed With Students 








* 


0 


0 


0 


26 


0 


0 


0 


0 


01 


0 


• 

0 


21 


Other Comments 


2 




0 


* 


2 


4 


3 


■'65'' 


3 


24, 


b 


60 




7 . 


2 


64 


Use of Results 


.* 


* 


* 




'5 


0 


8 


0 


12 


6 


n 


a 


7 4 


1 


8 


0 - 


Totals (1980 N«l^) (1981 H'-Mi) (1982 ii-64) 




26 


12 




4a 


24 


18 


11 


SI 


26 


?5 


>1 


8i 


2S 


IS 


6 
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the testing results and additionally 22% concerned the validity of the / 
findings. p 

Across the three years of the project, over half of the comments 

■ ■ ■, 1 
were critical of psychological assessment, 2S% wiere reconmendatlons for 

Improvement, 15/li were complimentary and only 6% were nonevaluative 

statements. The categories of written and/or verb'al comments addressed 

most frequently by reviewers concern^: (a) culture considerations Mn 

which 11 X criticized the reports, ^7% provided reconmendations for 

improvement, and 16X were complimentary of the attention given to 

cultural factors, (b) the findings in which 14X were critical of the 

findings section, 8X provided a recommendation for Improvement, and IIX 

were complimentary of the findings section, and (c) quality of the 

reports in which 33X were complimentary and only ^X was critical of the 

quality of the reports. \ 

The data presented in Table 16 and the analysis of the verbal and 
written comments suggest jth at as the study proceeded, the staff reviewing 
the reports focused on particular elements of each category and issue. 
In 1980, nearly all of the complimentary cotiments addressed the quality 
of the reports.^ In 1981 and 1982, when the quality of the reports was 
belrfg compa«8d with the earlier reports of this project, the complimen- 
tary comments covered a variety of categoreis and were more specific. In 
1980, most of the comments were general in nature, but In 1981 and 1982, 
reviewer comments Identified technicalities and specific itfems in the 
reports to comment on. ' 

The data contained in the Summary Tables 6, pa^e ^1, 7, page 59, 
8, page 65, and 9, page 69, a)l suggest positive trends in the "ab/lixty 
of the testing report" to communicate test findings and recommendations 

. .; -95 



more effectively^ to users. Table 4 SM^gests that the extent testing 
reports ^re used Increased durln^the dourse of the study and signif- 
icantly mort school personnel lodloBted they^recelved needed testing 
Information from reports rather than ftm consultants. In addition. 
Table 5 Indicates that testing reports wertf not only used more 
frequerrtly, but for more sophisticated purposes. I.e., to identify 

recommendations and/or determine the nature and cause of learning 

' . ■ - 

problems. * 

The referral form on which school per-sonnel referred children for 
testing was the same all three years of the study. The Inservlce 
training provided by the agency focused on explaining the use of the 

« 

referral forms,' procedures for determining short-term goals, and ^ 
exercises In preparing good referral questions. 

Frequent criticisms of the referral process stressed: (a) the 
referral form was too long and too complicated, (b) there was too much 
delay between the time the child was referred and tests were adminis- 
tered, and (c) that children had to be referred by school personnel early 
In the school yearj. (This criticism concerned the date fpr detehnlning 
the number ,Qif' children eligible ••or special education.) ^ 

Notes were kept of Informal comments, suggestions, and recommenda- 
tions throughout the course of the study and \:hen tallied with the 
comments that comprise Table 16. These data suggest the following 
conclusions concerning the communication between psychologists and school 
* personnel. 

K School personnel would prefer to have tiavajo school psychologists v^o 
a. . ' ' . ' . ' ■ • 

they felt would be more familiar jind considerate of Important 

Culturti and language factors* 



2. The forms from which students were referred were too long, compli- 
cated, and the process of ref erring students moved too slowly^ 

3. There was Insufficient t\me allowed for school personnel to consult 
with the psychologists. 

4. Testing reports should be more readily available to school 
personnel. 

The most complimentary comments, both formal and Informal, provided 
by school personnel about the assessment process Included the following: 
K The psychologists were competent and took time to obtain Information 
from the school personnel. 

■ • 

2. The testing reports were comprehensive and complete with findings and 

I. « 

recommendations that were appropriate. 

3. The reports were well written and Individualized, v^ereas a computer 
" prepared Report would have stock statements used throughout. 

4^The stafflngs at the conclusion of testing were very positive and one 
of the best parts of the entire testing process. 

5. The opportunity was appreciated to give opinions and provide recom- 
• mendatlons on . improving the individual psychological assessment 

process.* 



r 



DISCUSSION AND RECOMMENDATIONS 

It was the purpose of. this study to Improve the usefulness and rele- 
vance of Individual psycho-educational assessment In making placement and 
program decisions for .Native American children. The study focused on three 
BIA boarding schools of the Fort Defiance Agency on the Navajo Reservation. 
Consequently, there are many organizational, physical, and historical 
variables unique to this setting and agency, thus to generalize the findings 
and recommendations to other BIA schools or public schools In general, has 
some risks. Nonetheless, the findings of the study and the experience gained 
through conducting the study appear to have application to other BIA schools 
and public Schools in general. This section will identify some of the 

I 

factors, organizationally, programmatical ly, and/or culturally unique to the 
setting in which the study was conducted, discuss the findings with regard to 
these variables.* and provide recommendations for* which action can be taken. 

During the c6urjp of the study, a variety of changes occurred in the 
agency, personnel, policy and dynamics. Some of the changes that occurred 
are reflected in 'the data, but others were masked by statistical procedures. 

To interpret the findings and determine recommendations that will 
Improve the relevance and usefulness of testing data, the changes that 
occurred during the course of the study along with factors that could not be 
changed to accomnodate the study must be identified and discussed: 

^ ' (4' 

1. Dependent Variables , [ 

Only a limited amount of restructuring of various dependent vari- 
ables was possible/ The educational programs in the school sr' from which 
(fata were collected needed to remain intact,' thus certain activities 
of the study, i.e., scheduling for testing, inservice training, changes 
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in testing format, and jthe referral forms had to be arranged or main- 
tained in a manner that' would not violate th^ normal activities in the 
schools of educating chpldren. . 

2. Personnel Changes 

During the study, ihere were many changes in district personnel. 

similar to other BIA boarding schools, was 
approximately 40* of the faculty. This influenced the results of the 
study, for not all staffi had participated in the inservice training. 
Staff turnover also resulted in certain changes in educational philos- 
ophies and practices throughout the district and in the individual target 
school s. 

3. P. L. 94-142 Regulations and BIA Guidelines . \ ^ 

Department of Education and BIA regulations j^d guidelines for the 
implementation of Public Law 94-142 were not finalized or promulgated 

> 

before or during the: study. As a result, there were variations in the 

i 

interpretation regarjiing certain provisions such as timeframes, parental 
consent, parent participation, and confidentiality by different schools 
and individuals. Provisions of P^lic Law 94-142 were not violated, on' 
the contrary, the tendency was towards a very strict ''and narrow inter- 
pretation of such provisions. In certain cases, this interpretation 
seemed to interfere ^th the intent of the "Education for All Handicapped 
Children Act" by impeding the identification, placement, and provision of 
services to handicapped children. Furthermore, such interpretations were 
not always consistent for as personnel changed, inservice training was ■ 
conducted, and consultation was provided, certain changes were detected 
in the practices and interpretation of Public law 94-142 in the individ- 
ual schools. 
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Courrt Day 

The BIA Education Office changed the "count day" (the date 
eligibility for children in special education was determined for which 
additional funds were providecl through the U.S. Department of Education). 
This necessitated a change in scheduling students for assessment and lEP 
meetings. It also shortened the turnaround time from when a child could 
be referred to the time the testing reports had to be returned and lEP 
committees scheduled. To acconmodate these changes, additional individ- 
ual testing was scheduled 1^ the spring and diffe|;;ent times throughout 
the year. Throughout the study, efforts were taken to keep these 
variables consistent; however, the students in the Fort Defiance Agency 
are very mobile, and student transfers to and from the target schools 
were considerably greater than is generally experienced in elementary 
schools. The changes in the "count day" and students transferring from 
one school to another resulted in an undetermined amount of 
contamination. » 
Program Expansion 

During the study period, between 1980 and 1982, the agency experi- 
enced growth in the number of programs and services provided to handicap- 
ped children. Additional resource teacher^ were hired, additional 
self-contained units were established, and more psychologists and speech 
pathologists" were employed. This expansion in services and personnel is 
evidenced in the data collected to determine the impact of the study. As 
additioflal options for placement services became available, these options 
were more frequently recommended by psychologists and lEP committee 
members. Furthermore, the data collected suggest that school personnel 
in the target schools showed a positive change in: (a) their 
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understanding of psychological and special educational services, 

(b) their role and function as team members on the lEP coimittees, and 

(c) the role of the regul ar -teacher in providing special education 
services to children. Some of these changes could be interpreted as 
normal growth in sophistication and improvement in the agency, others 
were apparently stimulated by the activities of the study. 

Project Objectives and Suggestions 

In this section, the findings and conclusions for each project objective 

will be discussed along with suggestions for appropriate changes. 

Objective 1 « Strengths and Weaknesses In the Psycho-Educational 
Assessment Process of the Fort Defiance Agency \ 

The data collected to address project Objective 1 indicated three 
significant weaknesses in the psycho-educational assessment process: 
appropriate uti 1 i zat ion 'of -contract' psychological testing services, the 
referral process, and accessibility of testing reports. 
1 . Utilization of Contract Psychological Testing Services ' 

The Fort Defiance .School Agency, like many other BIA Agencies, 
has experienced difficulty in recrui'ting and maintaining profes- 
sional personnel such as special education teachers, and support 
staff' including psychologists, speech therapists, physical thera- 
pists, and occupational therapists. Contributing to this difficulty 
is the geographic remoteness of the area, lack of available housing, 
as well as cultural and environmental factors which are different 
and unfamiliar to most professionally trained people in these 
speciality areas. To obtain the professional services normally 
provided by specialty trained personnel, BIA agencies are sometimes 
forced to send children to off-reservation schools or institutions 
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in metropolitan ar^^ Another option includes entering into 
consortium agreements for speech therapists, occupational 
therapists, physical therapists, and psychological services which 
are then provided on a share-timed basis with other agencies. To 
obtain needed psychological assessment, BIA agencies have often 
entered into contracts with psychological service agencies. 
Contract psychologists generally travel to the schools on the 
reservation on a prearranged date, test 'the children referred, and 
return the completed psychological testing reports so they can be 
used in making placement and program decisions by lEP corrmittees. 
This procedure is, at best, a stop gap measure, for many of the 
other services such as consultation, inservice training, evaluation 
and direct intervention that would normally be provided by resident 
school psychologists cannot be provided through the typical testing 
service contract. Although it is less expensive to contract for 
testing services than to employ resident psychologists, such 
economic advantages must be compared with the need and availability 
of other psychological services which can only be provided by 

m 

resident psychologists. Comments recorded on the Test Critique 
Forms and provided In interviews with school staft pointed out that 
each year different groups of psychologists have been awarded the 
contracts to provide student assessment services. Each group uses 
different instruments, report formats, scheduling, and assessment 
procedures. f 

To maximize the use of contract psychological testing services 
two approaches are suggested: 

• # 
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a. Selecting and Monitoring Contract Psychological Test Services 
/ In reviewing past student assessment contracts, 1t was 
found that the selection of the contractors is often made with- 
out input from the school personnel who are to use the testing 
reports. Most often, the selection decision was made by 
business people in the area office; they determined who was 
selected on the basis of proposed costs and a license as 
evidence of proficiency. Selection based on cost encourages 
bidders to provide the minimum of services. They may administer 
the tests as rapidly as possible, utilizing an assembly Une 
approach which limits the interaction between psychologists and 
school personnel. In their efforts to reduce cost, they may 
also use computer generated reports which may not reflect the • 
- individual strengths and weaknesses of the student, individ- 
ualized recommendations, or the reality of resources available 
on the reservation. If testing reports are to be relevant and 
useful, cost must be balanced with the amount of interaction a 
contractor has with school personnel, and th^ psychologist must 
be'7amiliar with local resources and provide individualized 
' reports. To ensure that these provision's are provided, the 
contractor should be monitored and their process evaluated. 

The criterion of requiring a current state license to, 
practice psychology is not an appropriate requirement because 

i 

most state licensed psychologists are not necessarily prepared 
.to offer scjool psychological assessment services. The^ 
appropriate requirement should be a valid school psychologist 
certificate which ensures that the contract psychologist has 
completed appropriate course work and training in school 



psychology. Furthermore, to improve psychological services to 
Native American children, greater opportunities need to be 
provided for psychologists in training to obtain such €xperience 
under close supervision. y 
b. Utilizing an Established Testing Report Format 

The data in this study indicate the value of utilizing an 
established testing report format (see Table 10, page 73). 
Furthermore, the report; format needs to be updated, changed, and 
evaluated periodically to ensure that its content and length are 
reflecting information that is useful and relevant to the test 
report user (Rucker. 1967; Shively & Smith, 1969). 
The Referral Process 

The process used -to refer students for psychological testing 
was recognized, at the beginning and throughout the study, as a 
weakness. Even with inservice training, it appeared that the forms 
and the procedures were, in the view of school personnel, too 
complicated to facilitate referring children in a timely and 
appropriate manner. 

It is recommended that additional research and development "be 
undertaken on the referral forms to reduce and eliminate unnecessary- 
or duplicated information. It is suggested that the referral 
information be placed in packets with clear instructions to assist 
In Identification, screening, and .referring children for 
psychological testing (Chrlstenson et al., J982). 
Accessibility of Testing Reports and Confidentiality ^ 

Concern for confidentiality of testing reports appeared to be 
taken to an extreme in some of the schools. As a result, school 
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personnel felt they were discouraged from checking out or using 
testing data. The need for confidentiality is tantamount to the 
sensitivity of the material labeled as confidential. Properly 
prepared psycho-educatiolhil testing data does not need to be 
extremely confidential! Indeed, to be usable, primary emphasis 
should be on recommendations for intervention rather than on 
sensitive descriptive and causative factors. 

The testing report format used in this study was designed to 
focus on recommendations and the reports were written, edited, agd 
supervised with a full understanding that they would be presented to 
and discussed openly with parents. 

It is recommended that testing reports be made mr^ available 
to school personnel, that they be shared with parents, and that 
school psychologists prepare repor1% that are designed for suchr 
general use. 

Objective 2 - Provision For Inservlce Training of School Personnel and 
E CC Psychologists ' 

«| 

The evaluation data of the workshops and seminars provided as part 

of thl^ study focused on the opinions of participants. No systematic 
'attempts were made to measure knowledge or skills acquisition of 
participants. Changes that occurred during this study occurred 
for a variety of reasons. One may have been the inservice training. 
Data eval^uating the workshops, ilong with the informal comments and 
discussions with participants, emphasized that school personnel feel 
that they are "overly inservice trained". School personnel expressed 
concern that supervisors and administrators participated very little and 
that many other school personnel who needed the inservice training found 
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excuses to not attend the workshops. School personnel, particularly 
teachers, resented inservice training being imposed from the administra- 
ijkn without their participation in the topics seTi(c|ed, schedule, and 
methbd of presentation. Evaluation data indicate tff^t participants 
valued the simulated exer^i^e^ demonstrations, and small group 
participatory discussions much more than video-tapes and lectures. 

A variety of logistical problems were experienced in coordinating 
the school personnel workshops; these interfered with the smooth, 
wel 1 -organized task directed workshops that had been planned. In order 
to control cost, participant comforts such as air conditioned 
facilities, comfortable chairs, and coffee were eliminated. As one 
participant indicated, the school personner workshops were conducted' ^' on 
the cheep". * 

When awarding contracts for future inservice training on the 
reservation, it is suggested that cost containment not 'be the 
controlling factor. Provisions should be made for the workshops to 
expand the opportunity for participants to interact informally with 

A 

presenters, and that comfort, freedom 'from distractability and small 
group or individual activities be expanded.' 

ft ■•»-*- ■ . 

Objective 3 - Determining the Impact of Intervention Training 
Provided by the- Study """^ '•■ : 7^ 

Data 'to. determine the impact of intervention and training provided 

by the study was collected at the beginning, at midpoint, and at the 

conclusion of the project. These data indicate; that a number of <;hanses 

did occur reflecting the impact of the study. Sotne of these c^ianges 

Were determined to-be, statistically si gnifican^Tpand, others, although 

not significant, clearly indicate a„ positive trend. The data suggest 



that the testing reports were rated more useful and relevant vi^ien an 
abstract of the report was added at the beginning afong with surmiary 
sections. The ratings also improved when the psychologists expanded on 
the recomnendations and made' them more specif ic% 

One major limitation c6"ntracting psychologists must overcome "is 
their unf amil i arity with existing services. They seldom have sufficient 
contact with the district to become acquainted with the available 
resources, the strengths and weajcnesses of different programs and 
personnel, the instructional materials utilized, and the philosophy of 
instruction that exists. Each day following testing, a staffing of the 
children tested was held to discuss test findings and recommendations. 
During the staffings, the psychologists made notes on the coinnents of 
school personnel about services and materials that were available. This 
information was shared with other psychologists 1n debriefing sessions 
and then summarized and discussed in the seminars. This increased 
awareness of the available resources ^d hof^they- were valued reflected 
1n the recommendations section of future testing reports. The more 
favorable evaluations of the usefulness and relevance of the 1982 
recommendations section of the reports suggest that this had a positive 
Impact. 

The impact of the inservice training provided to school personnel 
was positive in some aspects, but not in others. Even though 
information and skills in preparing better referrals were stressed in 
both workshops, the referrals at the end of the study were judged as 
less adequate than they were at the beginning. However, there wa^s a 
trend that suggested school personnel were using testing reports more 
frequently and for more sophisticated purposes. Furthermore, at the end 
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of the study, 15X of the lEPs were judged to contain reconmendations 
transferred almost directly from the testing report. The use of testing 
reports to assist 1n placement and programning decision was a major 

» 

emphasis of the two workshops. , 

The opportunity to dialogue witK the psychologists and the data 
collectors was highly valued by school personnel for they often 
expressed their appriclation for the chance to express their opinions' 
.and provide recoirmendations concerning psychological assessment. During 
the sessions in which reviewers cri.tiqued the testing reports, they had 
the opportunity to "walk through"- the various sections of the reports. 
This appeared* to be a me^mTngfJl learning experiepce. Informal comments 
suggested that reviewers liked evaluating specific verbatim items from 
the reports. During these critiquing sessions, it was noted that the 
reviewers Increased in their sophistication as the study proceeded for 
the individual comments on the Test Report Critique Form Indicated a 
tendency for reviewers to be more critically demanding of the testing 
reports. 

It was observed when analyzing the test report critique data, 
reviewers tended to develop a negative impression of the reports when 
specific items of information were not included. When psychologists did 
not state the rationale for a test selected, what the test was meant to 
provide, did not comment on background information or developmental 
history, or explain efforts to consider cultural and language 
differences to avoid bias, 'tl\U suggested to the reviewer that they were 
not adequately addressed. Such explanations were needed to insure the 
test report user that the iPsycho log 1st was cognizant of such factors and 
that they were not forgotten or disregarded. 
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The only systematic complaint of school personnel about the testing 
reports was their length. This complaint presented a dilemma, for on 
one hand the reviewers were asking for increased specificity in the 
/recommendations section, more background information and explanation, 
yet on the other haitd they were- critical of the increased length that 
occurred as a result of the increased specificity. The changes in the 
test report format made during the study did indeed lengthen the testing 
reports. Efforts were made to counterbalance this by eliminating the 
wordiness of reports, duplicative information and material, and 
utilizing more sunjnary sections (see Resource Manual, pages 373 through 

The length of the testing Veports was, to a large extent, 
determined by the individual writing styles of the psychologists. The 
seminars addressed/ report writing and editing skills in an effort to 
reduce the length^ 

The study indicates that the results section and the 
reconmendations section have the greatest application to school 
personnel. Test report readers generally referred to the abstract, the 
summary of the results, diagnostic statement, and recommendations. 
Future attention needs to be directed at the placement and focus of 
these sections to increase their effectiveness in reporting information 
and data. Methods of highlighting these sections and reordering them 
should also be considered.* Concomitantly, additional Investigation is 
needed to determine which sections could be eliminated to make the 
report shorter. Possibly the long-term objectives, the criteria of 
success, and j^ggested materials could be reduced and incorporated into 
other sections. 
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Recownendatlons 

In the following section, recommendations derived from the data and 
experience of conducting tbis study are presented. The recomnendations and 
discussion that follow are presented for the purpose of improving the 
usefulness and instructional relevance of psycho-educational assessment. The 
following recommendations are designed primarily for the Fort Defiance Agency 
but have application as Well t6 other BIA boarding schools as well as public 
education in general. 

1 . It is recommended that the process and forms by which students are 

referred for psychological testing be examined and reevaluated to ensure 
that they contain provisions whereby the referral question can be clearTy 
Identified enabling the psyehologisls to respond to the Information needs 
of the referer . 

The data in this study suggest that the information teachers receive 
from psychological testing reports is consistent in specificity to the 
information they request. A vaguely or poorly .stated reason for 
referring ^ student for testing results in a testing report that contains 
equally vague and unspecific recommendations. This relationship seems 

analogous to referring an automobile to a mechanic and asking for a 

*> ■ * ■ 

complete "mechanical". With such a referral, not only is the mechanic 
likely to overlook the very problem necessitating the repair, but may 
also spend an inordinate amount of time determining what the user wants. 
Likewise, a psychological test referral that requests "a complete 
psychological" or "needs testing" does not identify a starting point 
permitting the psychologists to formulate assumptions a^bout cause or 
possible recommendations that can be validated with convergent testing 
data. When confronted with vague referral questions, the psychologists 
generally turn to using a stand3i*d battery of individual tests that 
probes in many areas without suffi^ent depth to obtain the information 
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of most value to the teacher. When the referral infonnation is specific, 
descriptive, and contains a clear statement of what the teacher wants to 
know and/or suggests what might be wrong,.- the psychologist is better able 
to provide meaningful data which is either new information- or confirms or 
refutes the teacher's suspicions (Mussman, 1964). 

Referral forms and procedures are often e^stablished with little 
attempt to systematical ly' evaluate them. The tendency is to include all 
of the background infonnation which might be useful rather than just the 
Information necessary. To collect or respond to all the information 
requested, the teacher must devote a great deal of time and effort. This 
complicates and lengthens the form without improving its usefulness. As 
a result, teachers either do not refer children that should be referred 
or they find ways to avoid completing the entire referral form.' Often 
the referral question, the presenting problem, or the specific 
information the teacher wants to know, is obscured by background and 
peripheral irtformatlon. 

This study suggests that inservice training alone is not Sufficient 
to improve the referral process. Teaching school personnel to identify 
good and clear referral questions, exercises in completing and writing 

* 

referral questions; completing referral forms, and walking through 
Instructions on forms may provide negligible if not negative results. 

Additional research is needed to ^alyze referral methods and 
processes. Such research must address the, design and evaluation of 
referral instruments and identify the procedures that will maximize the 
effectiveness while also minimizing the amount of time and effort needed 
to process the referrals. 
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2. To control cultural and language bias in psychological assessment, it is 
recommended that the procedures of converging data, obtaining second 
opinions, .and synthesizing reports be emphasized . 

Few Issues In education have engendered more debate than bias in 

' testing. -There are numerous books and articles which discuss the- pros 

and cons of the issues. Although the final chapter concerning bias in 

testing has not be wr^itten, a number of factors are suggested by the 

literature (Jensen, 1981; Duffey et al., 1981; Rosenbach & Mowder, 

1981): 

a. The primary purpose of standardized tests is to reduce or eliminate, 
' to the extent possible, subjectivity, or "bias", in decisfon making. 
The issue that caused education to use standardized tests is the 
same issue for which standardized tests are currently under attack. 
(Reschly, 1983; Kaufman, 1983; H^rnd, 1983). 
, b. Efforts to produce culture-free or culture-fair instruments have not 
been particularly satisfactory. For the most part, these instrufnents 
have faired little better than other standardized tests (Duffey et ^ 
al., 1981). 

c. Techniques, such as criterion-referenced tests or behavior checklists 
have' advantages, but they also have problems of sample selection and 
representation. Furthermore, when scrutinized for reliability and > 
validity, they fare little better than currently used standardized 
tests V( Duffey et al., 1981; Bailey & Harbin, 1980). 

d. Although unbiased Instruments' can and have been developed, they lack, 

predictive- validity; thus, they are not useful in predicting charjge 

\ \ 

and recomnendirlg Intervention (Bailey & Harbin, 1980). j . 

/ 
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e. Although there are some promising ideas for the future, at the 
present time, there are few acceptable substitute for using 
standardized tests (Clarizio, 1979; Linden, Linden, & Bodine, 1974). 
The above data suggest a measurement dilenina similar to the dilemma 
found by medicine in control ling the spread of infection. Even though 
some very powerful and effective disinfectants have been developed, their 
usefulness is 1im4ted for they often destroy the very thing that they are 
designed to protect. The fact that disinfectants do not control the 
spread of infection to everyone's satisfaction does not constitute 
grounds for banning their use". Furthermore, while research continues for 
disinfectants that are more effective and selective, at the same time, 

« 

procedural steps and safeguards have been initiated in most hospitals to 
monitor and control the spread of infection. ^ 

Procedural safeguards in the student evaluation provisions of Public 
Law 94-142 provide some broad procedures designed to minimize both 
subjective opinion and bias. in student assessment. 

It was not the purpose of thi^s ^udy to evaluate the bias in the 
testing; however, the opinions of test report reviewers and other school 
personnel suggest that the j)rocedural steps iniOfated and followed to 
control bias and' ensure cultural fai):;ness were favorably reviewed. 

Experimental research is needed to determine if such procedures do 
in fact control bias in testing or simply give such an illusion to the 
users." In the absence of such data, it is recommended that- the 
procedural steps followed in the dispositional assessment model , such as 
the one employed for this study, be adapted and utilized. These 
procedur'es include obtaining a second professional opinion on the 
diagnostic statements', test report findings, and recommendations; 
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selecting instruments that are as free from language and culture bias as 

possible; and testing causative and treatment hypotheses or assumptions 

through convergent measurement data (Mowder, 1983).' 

3. It is reconwended that a "standard" testing report format be designed 
based on the Information needs of test result users and provisions of 
Public Law ij4-l4ii . ~~ 

The single most important variable, v^ich t^st report reviewers felt 
enhanced the use and relevance of the testing reports more than any 
other, was the development and use of the Psychological Testing. Report 
Format (see Appendix C). The properties of the report format that 
reviewers evaluated as most helpful included: the flexibility it 
Vovided which encouraged the ind'ividualization of findings and 
recommendations, a format which ensured that the psychologists did 
address the specific information needs of the test report users and the 
provisions of Public Law 94-142. 

This study, however, as recommended by reviewers, suggests that the 
testing reports could be shorter. School personnel did not use all of 
the information in the reports. Techniques to highlight specific inform 
mation, sudi as the diagnostic statement and recomnendations, should be 
investigated. However, it is recommended that the report should continue 
to report orientation and background Information, including a description 
of the testing instruments and why they were selecfed, along with efforts 
to consider cultural factors. 

Future contractors that wll] provide psychological testing services 
on the reservation should use the established Psychological Testing 
Report Format or an adaptation thereof. 
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4. It is recomnended that Navajos fluent in their language and familiar with 
their culture be trained and certified as psychologists. Until competent' 
Navajo psychologists are available, the following are recommended: 
(a) Anglo school psychologists In training need to be given greater 
opportunities to work witn Native American children under appropriate . 
supervisioriy and (b) Navajo Interpreters must be carefully trained if 
they are to assist during psychological assessment . 

The data supporting recoirmendation No. 4 was derived from observa- 
tions and interviews conducted as part of the project. However, changes 
in the evaluation of testing reports suggests that the intervention 
initi-ated during the project address the concerns of school personnel 
interviewed. 

Many of the problems of recruiting and encouraging psycholoigsts to 
stay on the reservation could be addressed by training Navajos to provide 
psychological and assessment services to their own people. However ^ 
off -reservation training programs at university campuses have only been 
marginally successful. In such training programs, a large percentage of 
the Navajos at the completion of their training do not return to the 
reservation. Thus, it is suggested that training should be provided, at 
least p.art time, on the reservation itself. On-reservation training 
could include course work, practicum experience, and internships. It is 
essential, however, that reservation training maintain high standards of 
performance and close supervision. 

It is recognized that training and employing Nav^o psychologists is 
a long ranged- solution. As an interim, the' following are recommended: 
a* The relevance and usefulness of testing services provided by off- 
reservation psychologists was clearly enhanced by knowledge of 
certain cultural and language factors. However^ the only way off- 
reservation school psychologists can become sensitized and trained to 
'approp^'i ately consider cultural factors in their assessment efforts . 
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is to have the opportunity to work on the reservation under 
supervision. Current contracting practices requiring a license or 
school psychologist certificate curtails such arrangements. It is 
recommended that agencies on the reservation promote contracts with 
training programs that encourage the use of^sychologists In training 
to provide services on the reservation for practicum and internship 
experience. 

^ b. Psychological assessment of bilingual and/or pon-English fluent 
children presents problems that are diff icult^to resolve. As a 
pr*elim1nary step, v^en necessary and to the extent possible, 
non-language dependent tests should be- used. The majority of 
the children referred in this project demonstrated English langua'ge 
^ deficiences severe- enough to necessitate using one or a combination 

of unbiasd non-language dependent tests. However, utilizing 
oon-language dependent tests introduces problems of predictive 
validity,*''reli ability, diagnostic properties, and the usability of 
the data generated. Furthermore, non-language dependent tests are 
not effective in diagnosing academic skills, psycho-motor skills, or 
as a measure of social and emotional adjustment. To obtain measures 
in these areas, Navajo aides were available to interpret for the 
eb(aminer. For the 1980 testing, Navajo teacher aides were used 
following an orientation and introductory discussion that instructed 
them on how to appropriately interpret without adding or subtracting 
^ Information. For the 1981 testing, 13 Navajo students v^o were 
enrolled in a school psychology training program at Utah State 
University and spoke their nativ.e language fluently were utilized. 
These Navajo school psychology candidates were first used as 
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interpreters, then to interview for background data, and final to 
administer selected tests. During 1981 and 1982, interpreting 
directions and questions, and interviewing childre^ during testing 
was systematically improved. This improvement was recognized by 
school personnel, administrators, ECC psychologists, and data 
collectors. From this experience and information-, there was strong 
consensus -that interpreters cannot just be picked up because they 
have bilingual skills. If they are'^to contribute, they must be 
appropriately trained. Teacher aides and dorm personnel brought in 
on an incidental basis to interpret, in all probability, violate 
standardized^testing assumptions, thus destroying validity and 
reliability. 

Two other important factors relating to the use of interpreters and 
Ne^vajo testers were suggested that deserve consideration: 

First, it was observed that rapport with children was clearly 
affected by using iiiterpretfirs during testing. Children were often more 
shy and sometimes intimidated by the presence of another adult fluent in 
their native language.' Many of the normal rapport building techniques - 
that psychologists use such as "patter", "animation", and verbal 
reinTorcers became stilted and ineffective in the presence of a Navajo 
interpreter. Thus, the desired testing atmosphere of challenges, 
expectations, acceptance, and "gaming" was difficult to establish and 
maintain. " ; > 

Second, it was observed throughout the" study th^t in most cases, it 
was easier for Anglo psychologists to establish and maintain a positive 
task orientated rapport with the child than it was for the Navajo 
testers. When interviewing or testing children, the Najtajo school 
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psychology candidates took longer, and had to work harder to establiish 

^ and maintain rapport, motivation and ontask expectations than did the 

Anglo psychologists. When the Navajo school psychology candidates did 

establish optimal rapport, it was most often when they were using English 

rather than Navajo. Therefore, the assumption that fluency in the 

language facilitates rapport and ontask behavior, motivati-on, etc. was 

t 

not verified in this study. 

Three factors were identified which may account for these 
phenomena. 

^ a. The level of training and experience in psychological assessment was 
significantly higher for the Anglo psychologists than for the Navajo 
school psychology candidates. Furthermore, it was observed that, as 
the Navajo school psychology candidates gained experience, their 
skill in establishing and maintaining rapport and motivation with 
students improved. 

b. School personnel pointed out that Navajo students expect classroom 
instruction, directions, and discuftions to be conducted in English. 
As a result, they may respond to English instruction in a more task 
orientated manner than when instructions were presented in Navajo. 

c. It was suggested that cultural and social factors, how Navajps 
Interact trad it 1manyj*lthfiach other. iAJtemsjif^domin^^^ ^ „ 

expectation, and self disclosure may also accougt for the behavior 

r 

described above'. 

These observations and hypotheses need further Investigation for 
• which additional "research i$ suggested. 




It is recommended that future psycho-educational testing contracts 
include the following: (a) an established testing report format, 
(b) staff ings and increased systematic inter actio n between tester s and 



(b) staff ings and increased^ systematic interaction between teste rs an( 
school personnel, (c) orientation and training for the psycho 1 og"i sts, 
and jd) evaluation or the psychologist's performance ar>d procedures 
utilized by the assessment report usersT 7 



Recognizing thatlfemploying additional resident psycho"N)gists, 
especial ly Navajo psychologists, is a long-term solution, jit will be 
necessary to continue contracting off the reservation for psychological 
testing services. To enhance off-reservation assessment contractual 
services, the following are recommended: 

a. Utilizing an established testing report format. On page 103, the 
need for a standardized testing re"port format is discussed. The^ 
testing report format used in this study appears to be adequate, but 
additional evaluation data are needed on 'methods of improving this 
outline format. It is recommended that, a semi -standardized outline 
format be required in future contracts and efforts be undertaken "to 
systematically improve its usefulness and instructional relevance in 
placement and program development decisions. 

b. Staffings of chi Idren tested and systematic interaction between 
school personnel and diagnosticians. Throughout the study, it was 

--IL.^ , 

apparent that school W^onnel wanted increased opportunities to 
. discuss behavioral and instructional problems, recommendations, and 
placement decisions with the diagnostic personnel. Future contracts 

w 

shou,ld specify increased time for such interaction \ find dialogue 

' ' ' r 

between testing personnel and school personnel in both the collecting 
of assessment data and in discussing placement and program 
decisions. 

c. Orientation training for contract psychologists. This study suggests 

that the relevance of psychological testing reports and the usefiH- 
ness of^the placemeht and , program recommendations can be enhanced 
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when the psychologists are familiar with the services, instructional 

materials available, and the instructional philosophy of the school. 

It is recommended that future contracts for psychological testing ■ 

services make provisions to orientate the psychologists under 

contract with the services that are available in the agency, the 

instructional materials that are currently in use, policies and 

procedures followei^ and the instructional philosophy of the agency. 

A short orientation period, perhaps a half -day, is reconmended as a 

provision of future contracts. 

<1. To improve the relevance and usefulness of the individual assessment, 
'each dimension of the testing process should be systematically ' 
evaluated . Administrators or committees often make decisions 

concerning the content and procedures of making referraK, test 

report format and the assessment procedures. The recommend a^'ions 

and decisions of such committees, however, need' syst^atlc evaluation 

they are to be improved. It Is recommended that forms and 

procedures be subjected to periodic review and eval uation "and 

appropriate phanges be initiated based on evaluation data, ■ 

In addition, a systematic- evaluation „pl an ^should be- established to 

evaluate future psychological assessment contracts. This plan should 

include-in evaluation of the quality of. the reports, their' usefulness and;- 

accuffl^. It should evaluate the procedures fol lowed" and the. competence ' 

and helpfulness of the. psychologists, contracted. . . 

It is recommended that the Instructional prograih In the Fort Defiance . 
Agency lye reexamined to determine If sufficient emphasis is being placed 



on Instruction and drill anrf practice to ensure -mastery of 
in reading <and math for all c-ni idren . ~T^. * \. ,■ 
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Virtually every child referred for individuar testing, regardless of 

} .<■' ■' " ■ ■■ ' ' '" . " ' - ■"- 

agei. dernonstrated deficits in: (a) basic reading skills of word and ^ 
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letter recognition, phonics and word attack skills, and (b) basic math 
skills in addition and subtraction, combinations, counting and grouping 
as well as carrying and borrowing. Instructional deficits of t^ nature 
are generally caused by one or a combination of factors including: 

a. An instructional program that provides insufficient emphasis on basic 
skills mastery. Such programs may progress too rapidly through the 
basic slK^s^ They may not be appropriately sequenced or provide for 
sufficient drill and practice to enable the less capable students to 
master the skills before going to other things. 

b. Lack of mastery also occurs when there is insufficient monitoring of 
student progress. This permits students to "slide" by uptil they 
experience severe academic difficulties. 

»It is recommended that the curriculum be reexamined, along with 
instructional materials, methods, and monitoring provisions to determine 
if appropriate priority is being assigned to the mastery of basic skills 
by students. 
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Appendix 8 



Procedural Phases, Tasks, and 
Steps of the Study 



A 



^ ■ , ASSESSMENT RESEARCH STUDY 
Fort Defiance Agency 

Procedural Phases, Tasks, and Steos 
1 Januar)' 1982 



Phase One- -Assessment of the' Utilization and Educational Kelevance of Psycho- 
Educational Assessment Reports in the Fort Defiance Agency 

Pre-Project Activities 

Task A- -Collection of Baseline Data / 

/ 

Step 1-- Identification of the prijnary users of the 1980-81 individual psycho- 
educational testing reports in the Fort Defiance Agency. ^ 

Step 2- -Identification of the level and/or the extent the 1980-81 psycho- 
educational testing reports are used in the Fort Defiance schools. 

Step 3-- Identification of the types of information on the 1980-81 reports 
which the teaching staff felt to be most useful. 

Step 4- -Identification of the information on the 1980-81 testing reports 
which was seen by teachers and other school staff as being least 
useful . 

Step 5--Deteniiination of the relationship between the reconinendations and 
findings in the 1980-81 psycho- educational reports and the recom- 
fnendations contained from the students' lEPs. 

Step 6- -Determination of how valid the teaching staff at the Fort Defiance 
Agency feels that the reconinendations and findings on the 1980-81 
testing reports are. This will be undertaken by conroarison-and 
calculations of the perceptions of the teaching staff in review- 
ing testing reports. • 

Step 7- -Collection of infbimation on the 1980-81 psychological reports 

from the teachers at the Fort Defiance Agency as to hw^the report 
can be ixnproved in wording, format, presentation, length, etc. 

Step 8- -Analysis of the above data. 

Step 9- -Submission of the progress report, Phase One, Task A (June, 1981). 

Task B- -Development of In-Seivice Training Workshops for (a) Fort Defiance 
Staff, and (b) School Psychologists Selected to Provide Psycho- 
Educational Testing in September, 1981 to the Fort Defiance .\gency. 
(These workshops will be supported by an in-service training and 
testing contract awarded by* the Fort Defiance Agency. These workshops 
and above activities are not part of the .Assessment Research Project 
budget. The workshop's content among other things, however, will address 
an orientation for the Tort Defiance staff and school psychologists of 
the pr^ect to ensure cpns.istency and understanding of the procedures 
and so forth vAien the project begins in September, 1981.) 

Step l--Develop objectives, procedures, manuals^ etc. for in-sen'ice train- 
ing at Fort Defiance Agency. 
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Step 2--Conduct Fort Defianccjji- service training. 

Step 3- -Develop objectives, procedures, manuals, etc. for iii- sendee 
training of psychologists. 

Step 4- -Conduct a training workshop for the psychologists. 

Step 5--Prepare an evaluation report of the in-service training at the 
Fort Defiance Agency (Tall, 1981). 



Ccmmencement of the Assessment Research Grant Proposal As 

Approved by U.S. Department of Education, Office 
^ of Special Education Rehabilitl^tion Services 

September 3, 1981 » 



»^ess 



Task C- -Start Up Activities for the Assessment Research Study 
Step 1- -Appropriate revisions^f time-frames and tasks. 

i 

Step 2--]iiployment of staff. 

Step 5- -Revision of the Psycho- Educational Assessment Foimat. 
Step '4 --Monitoring test administration. 

Step 5 --Preparation of the report and evaluation of psycho- educational 
-assessment . 

Task D- -Assessment of the Utilization and Educational Relevance of Psycho- * 
Educational Assessment Reports for the Fort Defiance Agency (1981-82 
Academic Year). 

Step 1-- Identification of the primary users of the 1981-82 individual , 
psychological testing reports in the Fort Defiance Agency. 

Step 2— Identification of the level and/or extent of the 1981-82 psycho- 
educational testing- reports were useji by the Bort Defiance Agency 
staff. 

Step 3- -Identification of the tyt)es of information on the 1981-82 psycho- 
educational reports which the Fort Defiance staff felt was most 
useful. 

Step 4- -Identification of the information on the 19^1-82 testing reports 
which teachers and other school staff at the Fort Defiance Agency 
fcflt was least useful. 

Step 5- -Determination of the relationship between the recommendations and 
findings on the 1981-82 psycho- educational reports and the recom- 
mendations contained on the student's lEP. 
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Step 6- -Determination of how valid the teachers and staff feel that the 
reconrienda:tions and findings of the 1981-82 testing reports are. 
This will be determined by the sample of the teaching staff re- 
viewing the reports and correlating it with other data. 

Step 7- -Collection of infomation from the 1981-82 testing reports as to 
how to improve the 'reports in terms of .format, wording, presenta- 
• tion, length, clarity, etc. 

Step 8- -Analysis of the above data. * 

Step 9- -Preparation of Progress Report, Phase One, Task D. 



II. Phase Two— Development of Inservice Training Programs to Improve the Utiliza- 
tion and Educational Relevance of Psycho- Educational Assessment 

Analysis and examination of previous data collected, including Progress Reports 

Task A--Inse:Ace Training for Fort Defiance Staff 

Step 1- -Develop objectives, procedures, materials, etc. for in-service 
training of Fort Defiance Agency staff. 

Step 2 --Preparation of training manual. 

Step 3- -Review of training materials and procedures by content specialists 
at the ECC and content specialists fron Fort Defiance. 

Step 4- -Preparation of in-service training evaluation materials, 

• ■* ' ' 

Step 5--Analysis of workshop evaluation and preparation of evaluation 

report. y - 

Task B-- In- Service Workshop for Psychologists Assigned td^^Provide Individual 
Testing, Fort Defiance Agency, August, 1982 

Step 1- -Design objectives, procedures, and materials for the workshop. 

Step 2- -Review by content specialists at the ECC of objectives and materials 
for the workshop. 

Step 3- -Evaluation workshop 

>'Task C- -Administration of Psycho- Educational Testing, Fall, 1982 Participating 
Schools Following Revised Procedures and Revised Format-*' ^ 

III. PJjase Three- -Evaluation of Project Impact 
*Tasl< A- -Determine Project Impact 

Step 1-- Identification of the .m'imar^' i^er? ofahe 19S2 individv.sl psycho- 
Yp- educational testing, ri^port 5 in/ the/ Fort Defiance Acency. 
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Step 2- 



- Identification of the level and/or the extent the 1982 psycho- 
educational testing reports are used in the Fort Defiance schools. 



Ste^ 3- -Identification of the t>'pes of information on the 1982 reports 
Which the teaching staff felt to be most useful. 

Step 4- -Identification of th^infoimation on the 1982 testing reports 

v*iich was seen by teachers and othe^: school staff as being least . 
useful. j' 

Step S--DeteTmination of the relationship betiveen the recommendations and- 
findings in the 1982 psycho -educational reports and the recojimenda 
tions contained from the students' lEPs. 

Step 6--Detennipation of how valid the teaching staff at the Fort Defiance 
Agency '/feels that the recommendations and findings on the 1982 
testing reports Ire. This will be undertaken by comparison and 
calculations of the perceptions of the teaching staff in review- 
ing testing reports. 

Step 7- -Collection of information on the 1982 psychological reports from 
the teachexs at the Fort Defiance Agency as to how the report can 
be improved in wording, format, presentation, length, etc. 

Step 8 --Analysis of the above data. 

Step 9--Submission of the progress report, Phase One, Task A (June, 1982). 
Task B- -Analysis of Data Collected 
Task C- -Preparation for Submission of Final Report • 

Dissemination of Findings 
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Appendix C 



Example of the Psycho-Educational . 
Testing Report Critique Form 
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BESr COi'Y AVAILABLE 



Ptycho*duc«tlon«l. Tilting Report Crltlqut form 



INSTOJCTIONS. p.VCho^uc.tIon.l f,t,ng report for 

form by placing « ch«cH (vO on th« lln« rwxt to th« 
«ttt«»tot th«t Is cloflttt to your opinion. 



StudMtt' 



SclKWit 



How clMTly did this report 8t«t« 
this student's tssting results 7 



Very ciser. | undwrstood sverythlng. 

^k>i$T9U\y c\9mr, th»rs w«-s very few 
' things I couldn't understend. 

Moderetely unclMr, there were several 

points I couldn't umterstend. 

Not et ell clear, there were many 
points I couldn't understand. 



Please note some examples of things 
which were unclear to you for 
the Interview: 



Was this report useful In 
deterMlnIng this student's 
placeaieot? 



Very Useful 
SoMeifhat uaeful 



Not useful 



How often did you find technical 
words or phrases which were not 
adequately explained? 



The frequent use of Jargon made the 
reporfextrediely difflcuft to 
understand. 

There was substentlel Jargon used 
which nade the 'report herd to 
understand. 

"Sowe Jargon was used, but the report 
was usually understandable. 



Please note the phases or Jargon you 
found confusing for the Interview (If 
you prefer, go through the report and 
circle theai In red>t 



Little Jargon was used. 



Ho»r does this report oonpare with 
other reports you heve seen In 
the past year? 



About the saaw. 
Worse than the others. 



Comments ! 



B^er than the others 



Do these reconwendatlons eddreis 
the questions raised by the 

referring teacher? 



Referral question well addressed, 
feferral question partly addressed/ 
Referrel quest |on not addressed. 



Please note tome apeclfic questions 
which were not addressed. 



Do you feel that the exanlner geve 

appropriate consideration to .social 
and cultural factors In this student's 
case? 



Completely appropriate consideration 
Partly appropriate conslderetlon given. 



Pl**^ note some 
specifics! 



Partly Inapproprlete conslderetlon 
given. 



Completely Ineppropr fate consider- 
ation given. 
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Ltittd b«lott art Mvartl of tti« 
lACtloni fro« this student's 
report along with thoir itatad 
objactlvoi. 

In coluim A •valuata how mI I 
tha taction of tha report mi 
Its objacttvai. 

In coluMo B avaluatfl ho« uiaful 
tha taction wai In planning thli 
itudantif aducatlonal progran. 

Placa a chackKark (v^'on tha Una 
corraapondlng to tha itataMant 
that li cloiait to your opinion. 



REFERRAL INFORMATION 



How Mil did th 1 1 faction 
of tha raport naat thaia 
itatad objactlvai? 



B 



How uiaful wai this taction 
In praparing this studant's 
aducatlonal prograw? 



^ 



s/ // // 

S$.- IK- 

1^.' i// !>/ 1"/; 



.1 



Objactlvasi 1. Prov I da background about studant 

(triba, yaar at lntar«iountaln» ate.) 

2. list rafarring taachar(s) and 
th«lr ooncarns. 

5. List racords ravlawad and Inform- 
ation obtalnad. 

4. List findings and ra c oaia n datlons 
of tcraaning coMilttaa. 



If 



it 

si 



BACXGROUNO INFORMATION 



ObJactHvasi 1. list parsonal Infortiatrdlk about 

tha studant, faMlly, tribC haalth, 
school, apaclal Intaraats^atc. 

2. List tha problant as tha studant 
saas than. 

S.' Stata tha studant's ability to usa 
English to cciMmjnIcata. 



BEHAVIORAL OBSERVATIONS 



Objactlvast 1. Dascrlba how studant cooparatad 

during tasting (rapport, Motivation, 
Intaratt, languaga, anjoymant, ate.) 
2. Nota any spaclfic strangths or 
diff Icultlas. 

■ U — 

PSYCHO*EDUCATIONAL FINDINGS 

Objactlvas: 1.^ Dascrlba tast,'what It Masuras and 
why It Is usad. 

2. Raport rasults'as rangas^ parcantllas, 

and grada placamants, - - ■ 

3. Raport parsonal Ity and social factors 
at ««ottonal Indicators. 

J 



SUMMARY 

Objactlvas: 1. List studant 's strangths. 

2. SuMttartza tasting findings. 



RECOMMENDATIONS 



Objactlvast 1. Racofiwand placaiaant catagory. 

2. List othar pUcamnt con^ldaratlons. 

3. List long and fthort*tani goals. 

4. Suggaat Inatruetlonal iMtarlals 
* and strataglas. 

9. Suggaat support sarvlcas. 

6. Suggaat a laaans of avaluating tha 
studant*s aducatlonal program. 



DIAGNOSTIC STATEMENT 

Objactlvast 1. Racow ma nd mat approprlata 

placamant catagory for studant 
and rafar to tpaclfic guldallnas 
or ragulatlons. 
2. Qualify statamant to Includa othar 
1^ Information to ba prov I dad by tha 
^placement conwlttaa. 
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lascd OA IM UttfKg diU» Ibt pivcHologlil 
liAl Mdt s«v«r4l CONCiUSIONS. blow art 
sOBi ftUUwms ateul llio ilvdiNt Ukaii 
dfroctly Iroii IM psycbologfsl's ropori. 

Plaaia rata aach stataMoi In ilia caiagorlas 

to iha ri)|lii by pUctng a ckeckwirk 

U Ilia box wlilch rapreaanis your optnlon. 



Alfred wai gtvun both the Peabody PIciura Vocabulary 
Test and tha Frinury tanguage (>ia&tlonnalra to assess 
his facilUy ulth tha Cnyllsh tanguaga. On the PPVT» 
he achtavcd a ptircenttia scora of 4. This Mas an 
aita aqulwalanij score of 7 yaars, 7 attHihs. On the 
Priaiary LanguaQa Ouostjoiiiiaira» which Is a sarlas of 
ttuasilons askimi hls prefcraiices for spaaking Matfajo 
or Cngllsh. ha Indkaiad that ha prafers spaakliig 
English but doas speaK both languagas. 

In ordar to understand Alfred's ability to dIscrlKin* 
ata sounds In the English lanijuaga, lia was gfvctn the 
U«iu«an Auditory OiscrlMlnaM^n Test. Ills scora on 
this test Indicated he hdd difficulty dlscrlnlnatlng 
sounds In the English Unguaga. 

Alfred was given the Raven Progressiva Matrices to 
asMSs reasoning ability. Ills scora placed hl« 
" between the 2ilh and 50th Rercantlle whan coayared to 
other children his ago taking this test. I.e. » wlthl^ 
an average ramje of reasoning ability. 

The Devalo|MHontal Test of Visual-Motor Integration, a 
paper and pencil copying task, was utilized 4o 
assess visual perception and noior behavior. Alfred 
obtained an ago equivalency score, of 11 years 9 
«Orith&. 

Tha flirst subtest was raldUg which co#slSlad of r 
latter-word Iduntlffcation, word attack; and passaoa 
' coMprehanslon. lie received a grade equivalent ofi.6- 
^in overall readjng skills. On the letter-word Ident- 
ification test, ha could Identify letters and r«ad 
three-latter words. ^nder word attack, he could 
sound consonants ^Ipgly. but was not certain of vow*l 
sounds or coaOilnatlons of consonants and vowels. 

Ilia ^^^^^ consisted of two areas, calcula- 

tion and applied vatlieMatlcs. lie received a nath- 
aMttcs grade equivalent of 

His written language skills, which ware assessed 

through a dictation section and a proofing section. 
liUfi'J hifH .il d M^ii'le equivalent of 1.6. 
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Can you umlersiaiHl uhat 
the psycholoqist aieans by 
this stateMiitl 





noes this stattilunt pro- 
vide you with new Infor- 
nation about the student? 




lastd on your Mperl«nce 
with this student, is 
this staleaunt correct? 



t 





1 i 

Was this stataMnt Useful 
In planning this student's 
educational proqraM? 





0 

O 




Oat of ih% purooMi of tigtlug tl to provide \% th« rocoMMnda Ion «i 4n 

IU:CONHiNOATIOII$ or tutfyufttloAt which My be 4pproprl«U tevtl of ipacl- 

htlpfu) In pUiwIiitt tilt studMt'i tduc^lloiul . J fUltyT 

8ro9rM. listed belOM are toM of the ipecff Ic " 
IC0MN£NDA1I(MS t^keii directly frOM the r«|iort. 
Pleeie rate each of thfti. 



^ — STvAliMtloii flndlMQt iMd ttftt ftiultt iMdlcitt that 
Alfred It ellttlbU for spec til ediicitloii tervlcet 
luiHer the category of leereleQ dUablMtiet^ 



A1 rre 



red My benefit froei one*to-oiie lettructloe to 
roflwdUte reading and written lanuuaoe deficits. 
This My alio bt accoMpllshed by having hl« work 
with a progr^MMd leirnliig syste«. - 

It .Is recoaiieuded that sliert-tcni goals for Alfrid 
Includii tht following: 

A., Increased attendance at school through tieha- 
vloril proyraMMlng. 

b. lncroas«d division skills (see STCf ProgrM - 
Sliort Division, H). 

c. Increased reading skills (see STEP Prograa - 
General Reiiillny Skills. 114. CdMprehenston 
of UrIlUn Naterlal , I/). 

d. Increased self-confldiBnce (sie STCf Prograai 
Self-Confidence, fft). 

It Is recoMMended that long-ter« goals for Alfred 
Include the following: 

a. Hastery of basic wath (addition, subtraction. 
^ Multiplication, division) skills. 

^ li. Hastery of spelling and slfht^word vocabulary » 
|.e.» conslsteift recognition of foor^ to five- 
letter words. 

Because there was no rtecord of a vision or hearing 
test In Alfred's file.- his poor sound dlscrleilnatlon^ 
and his close proxldty to test Materials, It Is 
appropriate that he receive both evaluations to de^ ' 
tenelAe If poor vision or hearing way be Interfering 
with Ms learning ability. v 
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•ased on your eMperlence 
with this student, does this 
recoMMMidatloA provide you 
with new infomatlon? 



a 



6tvM th« r«kOitrcat hcrt at 
IntenMuntain, how rtallttic 
It Ihts rMOMwndat ioAl 




I 



Mow useful was this recoei- 
nendatlon In developing 
this student's Individ- 
ualized education prograMT 
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Student : 
School : 
Examiner • 
Date Tested: 



Fort Defiance Assessment Research 
■lEP Summary Sheet 

Classroom Teacher: 
Resource Teacher 
Other lEP Committee: 



Beferral Question? ' Yes 
Nature of Referral: Academic 
Quality of the Referral: Poor 



No Source; 
Behavior Physical 
Fair 5ood 



How well did the lEP address the referral question? Poor 



Other: 



Fair 



Good 



RECOMMENDATIONS AND OBJECTIVES 



Placement : 
Comments : 



LD 



MR 



ED 



PH 



Other 



Short Terra Goals: 

Reading ( Mechanics/Comp . ) : 



Math ( Computat ion/Func . /Concepts ) : 



Language (Expressive/Receptive) 



Auditory Training: 



Social Skills (Behavioral/ Interpersonal ) : 



Learning Skills (Attention. Attendance » Etc.) 



Motor Development: 



OtheF ; 



Long Term Goals: 



Materials and Strategies: 



Supportive Services (Speech, Medical, Etc.): 



Evaluation Criteria: 



OVERALL RATING: 1 - Pbor 2 - Fair 

Completeness of the Report: 
Appropriateness of the Report: 
Defc.ree of Specificity* 
^■erall Impression: 



3 - Gooc^ 



ERIC 



139 
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Appendix D 



Individual Tests Used by 
Psychologists and Frequenci 



171 
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TESTS ADMINISTERED AND FREQUENCIE 




A. 


Ability Tests: 


1980 


1981 


.1982 




1. 


fhe Coloured Progressive Matrices (Raven's Color) 


36 


47 


108 




2. 


The Standard Progressive Matrices (Reven's Standard) 


11 


41 


58 




3. 


Hiskey-Nebraska Test of learning Aptitude 
(Hiskey-Nebraska) 


119 


20 


16 




4. 


Wechsler Intelligence Scale for Children (Revised) 
(WISC-R) 


• 12 
33^ 


27 


9 




5. 


Leiter International Performance Test (Leiter) 


29 


68 


S 




The Columbia Mental Maturity Scale (CMMS) 


5 


18 


15 


B. 


Achievement Tests:, 










1. 


Woodcock -John s(fn Psycho-Educational Battery Part 
II Achievement T$sts (Woodcock-Johnson) 


\c.b 


QC 
OD 


1 C \ 




2. 


Woodcock Reading Mastery (Woodcock Reading) 


1 1 


A 
*f 


u 




3. 


Key Math 


o 


O 


1 




4. 


Wide Rangt Achievement Test (WRAT> 


Jo 


Q 
O 


u 




5. 


Peabody Individual Achievefflent^ Test (PIAT) 


0 


14 


in 

< > ■ ■* 




6. 


Brigance Inventory of Early Development (Brigance-K) 
Brigance Inventory of Basic Skills (Brigance-E) 


0 


n 
U 

* 


0^ ■ 




7. 


Diagnostic Arithmetic 




1 




C. 


Language Tests 










1. 


Peabody Picture Vocabulary Test (PPVT) 


107 


'^8 


143, ; 




2. 


Quick Test (QT) 


38 


49 


.45 , ' 




3. 


Wepman Auditory Discrimfnation Test (Wepman) 


152" 


102 


184 ^ 

* ** 

«. - ' 


D. 


Psycho-Motor Tests: 










1. 


Draw-A-Person (DAP) 


' 144 


116 


192 




2. 


Developmental Test of Visual -Motor Integration (VMI) 


143 


118 


192 . • 




3. 


Bender Visual -Motor Gestalt Test (BVMG) 


1 


0 


P ' 
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E. Social and Adaptive Behavior Tests: 



1. 


TeacTier Rating Scale ; 


170 


128 


201 


2. 


AAMD Adaptive Behavior Scale 


32 


17 


18 


3. 


Berks Behavfor Rating Scale (Berks) 


6 


12 


28 


4. 


Walker Behavioral Scale 


4 


10 


26 




/ ' TOTAL NUMBER TESTED 


170 


128 


201 




Appendix E 



Psycho-Educational Test 
Report Format 



< 



CONFIDENTIAL 

Interview and Draft Report Form 
'Revised, 1981 









/ 




year 




month 




day 








i 




year 




month 




day 




/ 




1 




year 




month 




day 



Student: 

Date of Evaluation: 
Birth Date: 



Age: 

Grade: 
School : 
Examiner: 

Tests Administered: 

/ 

J,- 

ABSTRACT 

Reason for the Referral ^ 
Procedures Used to Examine the Chil^jiifd to Minimize Bias 
Findings 
Recommendations 

■I 



(list all tests) 



REFERRAL INFORMATION 
.Tribe . 



Primary Language 



Secondary Language^ 



Pareht£ in Home: Mother^ 
Foster Placement 



Father 



_ Adoption^ 



Brothers 



Sisters 



Family Position 
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175 



Pronounced Health Problems 

Vision Hearing I 

Medication s Hospitalizations ______ 

Retention Transition 

Special Help Received: Special Education 

Remedial Reading Tutorial „___ 

Previous Evaluation s (Date) (Examiner) 

Tests Administered 



Previous {Recommendations 



Problems the Student Perceives (e.g., academic, social, etc.) 
Interests (e.g., clubs, sports, etc.) 



BEHAVIORAL OBSERVATIONS 

Communication^skills, receptivity, accessibility, rapport, 
motivation, persistence] distractibllity, language skills, 
enjoyment of the tasks, Ishif ting from one task to another, 
any outstanding physicalj features, and specific difficulties 
and/or strengths. 



V 
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PSYCHQEDUCATIONAL FINDINGS 

Start each test with a new paragraph. Include a^ort state- 
ment as to what th9 test measures and* why it was i^ed. 



Language Dominance 



V 



Receptive Language (QT/PPVT - Report mental age) 



\ 



Intellectual Assessment 



> 



Report as categories, bands, percentiles, range, etc. Do not 
report IQ scores. 



Psyfcho-MQtbr SkilU 

i Perceptual - Motor) 
[Gross Muscle) 
SmalkMuscle}/ 
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> 



Reading Task Analysis 

Report grade levels. Describe both comprehension and word attack/ 
analysis skills. Strengths artd weaknesses. 



Math Task Analysis 



Spelling Task Analysis 



ft 
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Socio-Emotional Findings 



SUMMARY 



173 
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RECOMMENDATIONS 



i* Diagnostic Statenient 

This statement should be in the words used in the BIA 
regulations (see example). If student does not meet BIA 
guidelines for classification, so state. 



2. Other Placement Considerat ions 

E.g., small group, one-to-one. reduce reading level ^f 
material in regular classroom, any particular education«l 
placement considerations, behavior modifications, voca- 
tional programming. ' 

For students not qualified for special education, address 
■ the student's educational related problems, if any, in 
terms of what needs to be done in the regular classroom 
or any other resources needed 

, . ' ■. , . ■ 
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Short-Term (Annual) Goals 



An annual goal represents the achievement anticipated 
for the student over a period of one school year. This 
Is an ^educated guess or estimate of where the student 
will be at the end of one year, if a prescribed intervention 
program is followed. 

Many goals will take more than one year to accomplish. 
The annual goals should be viewed in a sense, as the short 
range goals leading to broader expectations that will enable 
students to achieve their maximum potential upon leaving 
school . 

The areas which may need to be addressed in making short- 
term goals include: Reading, math, visual-motor skills, 
writing and spelling, interest; motivation, social and/ 
or personiality changes. 

In this section, the tester should indicate the entry point 
where instruction should begin in a subject area, state 
the first few instructional objectives, and then the recom- 
mended sequence that should be followed (refer to the STEP 
Program). 



Long-Term Goals • " 

Long-term' goals are a projection of the achievement and 
levels of functioning anticipated for a student over a 
period of two to four years; longer in some instances. 
These will be of a more global nature than short-term 
goals, but should be based upon extensions of the short- 
term ^oals; 

These may include: academic achievemertt, social chants, 
functiorval education, health, vocattorial, etc. 
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Instructional Materials and Strategies 

Any suggested materials or techniques such as small group 
materials, AV materials, oral/written suggestions, etc. 



Supportive Services 

Counseling; formal or informal, teacher counseling. Address 
any concerns or referrals for vision, hearing, health, etc. 



1 



Evaluation Criteria for Success 



Mastery of academic skills, impuTse control, improvement 
of self -concept, mpre group' pjirticipation, acceptance of 
responsibility, independence, confidence, etc., decreased 
absenteeism, control over emotions. 



15^ 
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